A screening method and screening device for differentiating early glaucoma patients

Abstract

The application discloses a screening method and a screening device for distinguishing early glaucoma patients. The screening method comprises the following steps: wearing a VR device by a subject and calibrating a gaze point, displaying a pre-detection scene image to collect eye movement and head movement data for calibration, then displaying a detection scene image and collecting corresponding data. After the data is combined based on time points, it is substituted into a glaucoma-normal person classification model for evaluation, so that whether the subject has glaucoma characteristics is judged. The screening method and the screening device provided by the application can effectively distinguish early glaucoma patients by analyzing the eye movement and head movement characteristics of the subject based on a virtual reality device, establishing a characteristic model of glaucoma patients, and providing technical support for early screening.

Description

TECHNICAL FIELD

[0001] The present application relates to a screening method for distinguishing early glaucoma patients, and also relates to a corresponding screening device, and belongs to the technical field of medical auxiliary diagnosis. BACKGROUND

[0002] Glaucoma is a group of irreversible blinding eye diseases characterized by progressive damage to the optic nerve. Related studies have shown that compared with elderly people with normal vision, elderly people with glaucoma are 3-4 times more likely to fall within a year, and it has been confirmed that severe visual impairment is associated with falls.

[0003] The traditional glaucoma diagnosis method is fundus medical image analysis, which comprehensively detects and identifies the fundus lesions of patients by analyzing the shape, color and structure of the fundus of the patients. For example, in the Chinese invention patent with the patent number ZL 202410343063.0, a fundus medical image analysis method based on computer vision technology is disclosed. The method collects the fundus medical image of the patient, and introduces an image processing and analysis algorithm based on computer vision technology and artificial intelligence technology in the backend to analyze the fundus medical image.

[0004] Similarly, the prior art is to detect and identify the fundus lesions of patients by comprehensively analyzing the shape, color and structure of the fundus of the patients. However, in reality, the early symptoms of primary glaucoma are hidden, and only 10%-50% of the patients with glaucoma are aware of their condition, and 75% of the patients who seek medical treatment have developed to the middle and late stages of glaucoma. Therefore, early glaucoma screening is crucial. SUMMARY

[0005] The primary technical problem to be solved by the present application is to provide a screening method for distinguishing early glaucoma patients.

[0006] Another technical problem to be solved by the present application is to provide a screening device for executing the above-mentioned screening method.

[0007] To achieve the above technical purposes, the present application adopts the following technical solutions:

[0008] According to a first aspect of the embodiment of the present application, a screening method for distinguishing early glaucoma patients is provided, which comprises the following steps:

[0009] Step 11: The subject wears a VR device to calibrate the gaze point;

[0010] Step 12: Enter the pre-detection time, display the image of the pre-detection scene on the display module of the VR device, collect the eye movement data and head movement data of the subject when observing the image, and calibrate;

[0011] Step 13: entering the detection time, displaying the image of the detection scene on the display module, collecting the eye movement data and head movement data of the subject when observing the image;

[0012] Step 14: merging the eye movement data and the head movement data based on the time point;

[0013] Step 15: substituting the merged data obtained in the above step into the glaucoma-normal person classification model for evaluation to determine whether the subject meets the characteristics of glaucoma patients.

[0014] Preferably, in the step 13, all instance objects in the virtual reality environment are sub-objects of the VR device camera, so that the relative position of the eye position and the specified field of view range does not change.

[0015] Preferably, the step 13 includes the following sub-steps:

[0016] Sub-step 131: entering the fixation detection time, displaying the fixation detection content on the display module, and the subject gazes at the fixation detection content; the fixation detection content is an instance object in a static state within a 10° and 24° circular field of view range in a white and black panoramic background;

[0017] Sub-step 132: entering the tracking detection time, displaying the tracking detection content on the display module, and the subject gazes at the tracking detection content; the tracking detection content is an instance object in a moving state within a 10° and 24° circular field of view range in a white and black panoramic background;

[0018] Sub-step 133: entering the matching detection time, displaying the matching detection content on the display module, and the subject gazes at the matching detection content; the matching detection content is a plurality of instance objects in a static state within the field of view range in a white and black panoramic background.

[0019] Preferably, the panoramic background in the step 12 and the step 13 is white and black, and the white background and the black background appear alternately when the step is transformed.

[0020] Preferably, the step 14 includes the following sub-steps:

[0021] Sub-step 141: reading the eye movement data and the head movement data into a DataFrame table type data structure;

[0022] Sub-step 142: storing the timestamp difference of the eye movement data and the head movement data; storing the minimum timestamp difference found, and the initial value is set to infinity; recording the index of the last merged row in the head movement data, and the initial value is 0;

[0023] Sub-step 143: Traverse each row of the eye movement data from the 0th row to the mth row, m being a positive integer;

[0024] Sub-step 144: Traverse each row of the head movement data from the index-specified row to the ith row, i being a positive integer;

[0025] Sub-step 145: Calculate the timestamp difference of the current row; if the difference is less than the minimum timestamp difference recorded at present, update the minimum timestamp difference; otherwise, merge the mth row of the eye movement data and the ith row of the head movement data; reset the timestamp difference and the minimum timestamp difference, and update the index i of the current row.

[0026] Sub-step 146: Exit the sub-step 144 and continue the sub-step 143 until the eye movement data is traversed;

[0027] Sub-step 147: Save the merged timing data.

[0028] Preferably, the step 13 comprises the following sub-steps:

[0029] Sub-step 134: Evaluate the eye movement data in the detection time to obtain a task index;

[0030] Sub-step 135: Compare the task index with the data of the baseline experiment to determine whether the subject meets the characteristics of a glaucoma patient.

[0031] Preferably, the method for obtaining the glaucoma-normal person classification model comprises the following steps:

[0032] Step 21: Diagnose the eye health of the subject through medical examination;

[0033] Step 22: The subject wears a VR device to calibrate the gaze point;

[0034] Step 23: Enter the pre-detection time, display the image of the detection scene on the display module, collect the eye movement and head movement data of the subject when observing the image, and calibrate;

[0035] Step 24: Enter the detection time, display the image of the detection scene on the display module, and collect the eye movement and head movement data of the subject when observing the image;

[0036] Step 25: Merge the eye movement data and the head movement data based on the time point;

[0037] Step 26: Obtain sufficient data of the diagnosed subject to form a data set, and divide the training set and the test set of the data set;

[0038] Step 27: based on the 1D CNN model and the training set, training the glaucoma-normal person classification model to evaluate the model with the test set.

[0039] Preferably, the step 27 comprises the following sub-steps:

[0040] Sub-step 271: based on the 1D CNN model and the training set, adding LSTM or Attention layer on the basis of CNN to build 1D-CNN model, LSTM+1DCNN model, 1DCNN+Attention model and LSTM+1DCNN+Attention model for training;

[0041] Sub-step 272: obtaining model evaluation indicators based on the test set;

[0042] Sub-step 273: selecting the best model according to the evaluation indicators.

[0043] According to the second aspect of the embodiment of the present application, a screening device for distinguishing early glaucoma patients is provided, comprising a VR device and a control device; wherein,

[0044] The VR device comprises a display module, a head movement sensor, an eye movement sensor, an audio module and a communication module; the display module is used to display images; the head movement sensor is used to obtain head movement data; the eye movement sensor is used to obtain eye movement data; the audio module is used to play audio prompts; and the communication module is used to realize data transmission;

[0045] The control device comprises a computing module and a storage module, and the computing module reads the computer program in the storage module and is used to execute the screening method for distinguishing early glaucoma patients.

[0046] Compared with the prior art, the present application provides a virtual reality technology-based early glaucoma patient screening method and a corresponding screening device. The method can effectively distinguish early glaucoma patients and normal people by making the subjects wear VR devices and collecting a series of eye movement and head movement data. In the virtual reality environment, the eye movement and head movement characteristics generated by the subjects under different scenes and tasks are recorded and analyzed, and these characteristics are significantly different between early glaucoma patients and normal people. By combining the collected data and substituting it into the glaucoma-normal person classification model for evaluation, it can accurately determine whether the subject meets the characteristics of early glaucoma patients. The present application improves the accuracy of early glaucoma screening and helps to reduce the risk of irreversible vision loss caused by glaucoma. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1A In the embodiment of the present application, a white background scene diagram of the eye movement detection scene is shown.

[0048] Figure 1B Fig. 1 is a white background detection picture and a ball-out scene schematic diagram of an eye movement detection scene in an embodiment of the present application;

[0049] Figure 1C Fig. 2 is a black background scene schematic diagram of an eye movement detection scene in an embodiment of the present application;

[0050] Figure 1D Fig. 3 is a black background detection picture and a ball-out scene schematic diagram of an eye movement detection scene in an embodiment of the present application;

[0051] Figure 2 Fig. 4 is a screen recording schematic diagram of eye movement detection in an embodiment of the present application;

[0052] Figure 3 Fig. 5 is a distribution ratio schematic diagram of a training set, a test set and two types of labels of a data set in an embodiment of the present application;

[0053] Figure 4A Fig. 6 is a loss function decline curve schematic diagram of a training set in an embodiment of the present application;

[0054] Figure 4B Fig. 7 is a loss function decline curve schematic diagram of a test set in an embodiment of the present application;

[0055] Figure 5A Fig. 8 is a two-classification confusion matrix schematic diagram of a 1DCNN model in an embodiment of the present application;

[0056] Figure 5B Fig. 9 is a two-classification confusion matrix schematic diagram of a 1DCNN-Attention model in an embodiment of the present application;

[0057] Figure 5C Fig. 10 is a two-classification confusion matrix schematic diagram of a LSTM-1DCNN model in an embodiment of the present application;

[0058] Figure 5D Fig. 11 is a two-classification confusion matrix schematic diagram of a LSTM-1DCNN-Attention model in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The technical content of the present application will be described in detail below in combination with the drawings and specific embodiments.

[0060] The technical concept of the embodiment of the present application is to identify early glaucoma patients by using the eye movement and head movement characteristics of glaucoma patients in a virtual reality environment. In the early stage of glaucoma, although the central vision of the patient may not have been impaired, their peripheral vision has already been abnormal. In order to compensate for the decline in visual function caused by this visual field damage, the patient will unconsciously exhibit different eye movement and head movement behaviors from normal people. Therefore, by analyzing these behavior characteristics in a specific virtual reality scene and task, early glaucoma patients can be distinguished because these characteristics are significantly different from the normal population. In short, the present application realizes the screening of early glaucoma by identifying the differences in eye movement and head movement behaviors between early glaucoma patients and normal people.

[0061] In the embodiment of the present application, we use the HTC VIVE Pro Eye VR headset (referred to as VR device) as the detection device. The software system is developed using the SteamVR plug-in, which enables the system to process input and output information related to the VR controller and load the 3D model of the VR controller. In order to collect eye movement data, we use the SRanipal software development kit (SDK), which can collect directional vector information of eye movement; and the head movement data is collected through the OpenVR API, which can obtain quaternion data of head movement. In addition, we have built a human motion experiment system based on Unity3D, which is specially used to build a detection platform for glaucoma patients to analyze eye movement and head movement.

[0062] First embodiment

[0063] The first embodiment of the present application provides a screening method for distinguishing early glaucoma patients, which at least includes the following steps.

[0064] Step 11: The subject wears the VR device and calibrates the gaze point.

[0065] As mentioned earlier, the embodiment of the present application uses the HTC VIVE Pro Eye VR device as the detection device. The VR device includes a display module, an eye movement tracking device, and a head movement tracking device. After the subject correctly wears the VR device, the face is directed straight ahead, without lowering the head, tilting the head, or turning the head. The operator adjusts the position of the VR device according to the comfort of the subject to ensure that the display screen of the VR device is within the line of sight of the subject and the image is clear. Adjust the interpupillary distance (IPD) to ensure that the image is aligned with the eyes.

[0066] After wearing the VR device, enter the calibration interface and complete the calibration through a series of visual prompts. The visual prompts include a series of dots or target points on the display module, and the subject gazes at these points for calibration.

[0067] Optionally, the display module displays a calibration point (e.g. a dot or a cross mark) and prompts the subject to gaze at the calibration point for a certain duration (e.g. a few seconds). Subsequently, the calibration point moves to different positions (e.g. five to nine points) and the subject gazes at each point in turn.

[0068] The eye tracking device (e.g. Tobii eye tracker) detects the subject's eye position and eye movement through infrared sensors and cameras, and captures the subject's gaze direction and fixation point. The eye tracking device tracks the subject's eye movement data in real time, calculates the point in the display module or three-dimensional space that the subject is gazing at, and determines whether the subject's gaze is in contact with the instance object.

[0069] IGazeFocusable is an interface defined in Tobii SDK that allows us to handle gaze events related to eye tracking in the Unity environment. With this interface, the operator can pre-set a series of behaviors that will be triggered when the subject's gaze focuses on a specific object or leaves the object. In the IGazeFocusable interface, there is a core method called GazeFocusChanged (bool hasFocus), which is called internally by Tobii SDK. When the eye tracking device captures the subject's gaze entering or leaving an object, the GazeFocusChanged method is triggered to execute.

[0070] The GazeFocusChanged method accepts a boolean parameter hasFocus, which is provided by Tobii SDK to indicate whether the current object is being gazed at by the subject. The value of hasFocus is updated in real time as the subject's gaze changes. Specifically, when the subject's gaze focuses on an object that implements the IGazeFocusable interface, the GazeFocusChanged method is triggered and the hasFocus parameter is set to true, indicating that the object is being gazed at. Conversely, if the subject's gaze moves away from the object, the GazeFocusChanged method is also triggered, but this time the hasFocus parameter is set to false, indicating that the object is no longer being gazed at. This mechanism allows developers to perform corresponding operations based on the focus and movement of the gaze.

[0071] Step 12: Enter the pre-detection time, display the image of the pre-detection scene on the display module, collect the eye movement data and head movement data of the subject when observing the image, and perform calibration.

[0072] As Figures 1A-1DAs shown, the pre-detection scene is set as a white or black panoramic background to test the difference in the performance of the subject under different brightness backgrounds. Among them, the image of the detection scene is taken as the background by the sphere surrounding the VR camera (the white or black panoramic background is pasted on the sphere), and the indication information (such as arrow map) is displayed on the background, and the arrow points to the central detection area to guide the subject to concentrate the line of sight on the central detection area. Among them, the map attribute of the arrow is double-sided coloring.

[0073] This step is a pre-detection step, which is a complete eye movement experiment of the eye movement scene (black panoramic background or white panoramic background). If the drift or accuracy of the fixation point is observed, the VR device re-performs eye movement calibration on the subject.

[0074] It is worth noting that the pre-detection time is used to let the subject familiar with the detection content and the device, and to ensure that the subject correctly understands the task requirements. Therefore, the detection scene image displayed by the pre-detection time has similarity and continuity with the image of the subsequent detection.

[0075] The collection of eye movement and head movement data is divided into a waiting phase and a detection phase. The waiting phase refers to the time between the end of the previous detection task and the start of the next detection task. When the VR device gives a start instruction, the subject fixates on the image in front of him and gives feedback (such as pressing a switch), and the detection phase starts. When the VR device gives an end instruction, the subject fixates on the exit icon and gives feedback, and the detection phase ends. When collecting eye movement and head movement data, the program records the start and end times of the waiting phase and the detection phase to distinguish between the two phases. The start and end times of the detection are determined by the line of sight and the feedback of the subject, and both must be true at the same time to control the detection process.

[0076] Optionally, during the detection process, Instantiate() function is used to generate target instance objects at random positions within a specified range, and if the eye movement tracking device detects that the line of sight of the subject contacts the instance object, a timing function is triggered. When the contact time exceeds the set threshold, it is determined that the target object has been successfully eliminated.

[0077] Optionally, the Tobii eye tracker built into the VR device captures the line of sight through the Tobii XR SDK, and the prefab of the instance object is assigned the same size of the collider attribute, which specifies that when the line of sight intersects with the instance object (that is, when the fixation is to the instance object), the relevant instance object is marked with a red mark, and the background log records the start and end times of the fixation. When the fixation reaches the required duration and there is no interruption, it is considered as a successful eye movement behavior.

[0078] Since head-eye coordination movement will be generated in the process of visual line transfer, the interference of head movement on eye movement detection needs to be excluded in the design of detection scene. The specific implementation is to take the target instance object in the detection scene as a child object of the VR camera, and the world coordinates of the child object are the sum of the world coordinates of the parent object and the coordinate vector of the child object in the coordinate system of the parent object. In other words, when using the VR device to perform eye movement and visual field test, all instance objects in the virtual reality environment are taken as child objects of the VR camera, so that the relative position of the eye position and the specified visual field range does not change, and external head movement and body movement will not affect the eye movement detection result.

[0079] It is worth noting that the instance object provided by the embodiment of the application is a figure in the detection scene that needs to be gazed at by the subject. Although head movement data is still important in subsequent test steps, head movement interference needs to be excluded in order to obtain eye movement data, so it is collected together.

[0080] Step 13: Enter the detection time, display the image of the detection scene on the display module, and collect the eye movement and head movement data of the subject when observing the image.

[0081] The detection scene is set as a white or black panoramic background, and two formal detections of the white panoramic background and the black panoramic background are completed in a random order in this step. The white panoramic background can be detected first, or the black panoramic background can be detected first in a random order.

[0082] It is worth noting that if the white panoramic background is selected in step 12, step 13 needs to detect in the black panoramic background first, and then detect in the white panoramic background. Conversely, if the black panoramic background is selected in step 12, step 13 needs to detect in the white panoramic background first, and then detect in the black panoramic background.

[0083] In the black panoramic background and white panoramic background experiments, the subject completes a plurality of eye movement tasks in turn. For example, 10° fixation (30s), 24° fixation (30s), 10° smooth pursuit (30s), 24° smooth pursuit (30s), and visual matching (60s) are completed in the black panoramic background and the white panoramic background respectively, a total of ten eye movement tasks, as shown in Figure 2 The black panoramic background and the white panoramic background are displayed alternately.

[0084] The virtual reality eye movement detection is guided by the eye movement task and is designed with reference to the measurement principle of the perimeter. For example, the instance objects appear in a certain distance and a certain visual field range in front of the screen in turn, and the subject needs to complete the tasks of fixation, saccade, visual matching and the like on the instance objects.

[0085] For example, in Unity3D, a plane is defined at z = 5, and the z coordinates of all instance objects are fixed at 5, so that they are all on the plane. The VR camera is equivalent to the human eye, the plane is equivalent to the "screen", and the randomly appearing instance objects are equivalent to the "test light points" of the visual field meter.

[0086] This step includes the following sub-steps.

[0087] Sub-step 131: Enter the gaze detection time, display the gaze detection content on the display module, and the subject gazes at the gaze detection content. The gaze detection content is an instance object in a static state within a 10° and 24° circular visual field range.

[0088] The instance object is exemplified by a small block of 0.2m x 0.2m x 0.01m, and the visual field range includes a 10° and 24° circular visual field range. An instance object randomly appears within the visual field range, and if the gaze on the instance object lasts longer than the set threshold (e.g., 1s), the instance object is eliminated, and the next instance object appears after a certain time interval (e.g., 1s). If the instance object is not eliminated (the gaze duration is not maintained) for more than the set threshold (e.g., 3s), the instance object immediately disappears, and the next instance object appears after a certain time interval (e.g., 1s).

[0089] Sub-step 132: Enter the tracking detection time, display the tracking detection content on the display module, and the subject gazes at the tracking detection content. The tracking detection content is an instance object in a moving state within a 10° and 24° circular visual field range.

[0090] The instance object is exemplified by a small block of 0.2m x 0.2m x 0.01m, and the visual field range includes a 10° and 24° circular visual field range. An instance object randomly appears within the visual field range and uniformly translates within the visual field range at a speed of 0.8m / s. If the gaze on the instance object lasts longer than the set threshold (e.g., 2s), the instance object is eliminated, and the next instance object appears after a certain time interval (e.g., 2s). If the instance object is not eliminated (the gaze duration is not maintained) for more than the set threshold (e.g., 5s), the instance object immediately disappears, and the next instance object appears after a certain time interval (e.g., 1s).

[0091] It is worth noting that the implementation object that maintains the gaze and uniformly translates is referred to below as smooth tracking.

[0092] Sub-step 133: Enter the matching detection time, display the matching detection content on the display module, and the subject gazes at the matching detection content. The matching detection content is multiple instance objects in a static state within the visual field range.

[0093] The example objects are exemplified by a plurality of mutually different graphics, for example 8. The plurality of example objects are regularly arranged in the field of view, for example in a circular arrangement and a rectangular arrangement. A randomly selected example object appears repeatedly in the central region of the arrangement of example objects. The subject finds the example object in the surrounding example objects that has the same graphic as the central example object, and maintains the gaze for a certain period of time (for example 1s), and the example object in the central region is eliminated. After a certain period of time (for example 2s), the next example object appears in the central region. If the set threshold (for example 5s) is exceeded without eliminating the example object (failing to maintain the gaze for a certain period of time), the example object disappears immediately, and the next example object appears after a certain period of time (for example 1s).

[0094] It is worth noting that the surrounding example object with the same graphic as the central example object is referred to as matching.

[0095] Step 14: Merge the eye movement data and head movement data based on time points.

[0096] Because of the natural coordination mechanism of the human eye and head, the head movement data also contains valuable information for detection. The role of head movement data in behavior analysis mainly reflects in the following aspects.

[0097] Gaze stability: Head movement affects the stability of eye movement, and analyzing the amplitude and frequency of head movement can help understand how the subject adjusts the gaze during viewing.

[0098] Attention shift: Even in the case of head stillness, head movement data still reflects subtle changes in the subject's visual attention, such as brief head micro-movements indicating attention to a specific area.

[0099] Individual differences: Different subjects have different patterns of head movement when viewing the same content, and this difference can distinguish between glaucoma patients and normal people.

[0100] Environmental adaptability: Head movement data can analyze how the subject adapts to visual information in a virtual environment, providing a more comprehensive understanding of behavior.

[0101] In summary, the value of head movement data in behavior analysis lies in supplementing eye movement data and providing a more comprehensive understanding of gaze and attention.

[0102] Because the VR device collects eye movement data and head movement data from devices with different technical principles, the sampling frequencies of the data are different, and the specific time labels of the two data cannot be one-to-one corresponding. It is necessary to merge the two parts of data based on time points to create a basis for subsequent processing.

[0103] For example, the timing data collected by the embodiment of the application includes both eye movement and head movement data, and the experimental data is recorded at a set sampling frequency. The timing merging algorithm processes the data for down-sampling to match the minimum sampling frequency, so as to ensure that the two kinds of data are aligned with respect to the minimum time difference. After the data is merged, it is checked whether there are missing values or abnormal points, so as to ensure the quality of the data.

[0104] As shown in Table 1, the timing merging algorithm steps are designed according to the timing merging of eye movement-head movement in the embodiment of the application. The eye movement data and the head movement data are input in the form of a csv file, and the merged timing data is output in the form of a csv file.

[0105] Table 1 Timing merging algorithm

[0106]

[0107] Specifically, the step includes the following sub-steps.

[0108] Sub-step 141: read the eye movement data and the head movement data into a DataFrame table type data structure.

[0109] The eye movement data (timing data 1) and the head movement data (timing data 2) are read into a DataFrame table type data structure.

[0110] Sub-step 142: store the difference between the time stamps of the eye movement data and the head movement data; store the found minimum time stamp difference, and the initial value is set to infinity; record the index of the last merged row in the head movement data, and the initial value is 0.

[0111] Initialize the time stamp difference time_difference and the local minimum value min_time to infinity, and the last merged row number memo = 0.

[0112] Sub-step 143: traverse each row of the eye movement data from the 0th row to the mth row, where m is a positive integer.

[0113] Sub-step 144: traverse each row of the head movement data from the index specified row to the ith row, where i is a positive integer.

[0114] Sub-step 145: calculate the time stamp difference of the current row. If the difference is smaller than the currently recorded minimum time stamp difference, update the minimum time stamp difference. Otherwise, merge the mth row of the eye movement data and the ith row of the head movement data. Reset the time stamp difference and the minimum time stamp difference, and update the index to the current row index i.

[0115] Calculate the time stamp difference time_difference = abs(time_stamp_1[m]-time_stamp_2[i])

[0116] if time_difference < min_time: update min_time = time_difference

[0117] else: merge the mth row of data 1 and the ith row of data 2, reset time_difference, min_time, memo = i.

[0118] Substep 146: exit substep 144 and continue substep 143 until the eye movement data is completely traversed.

[0119] break loop 4, continue loop 3, until the rows of data 1 are completely traversed.

[0120] Substep 147: save the merged time series data and output as a csv file.

[0121] Save the merged DataFrame and output as a csv file.

[0122] Step 15: substitute the merged eye movement data and head movement data obtained in the above steps into the glaucoma-normal person classification model for evaluation to determine whether the subject meets the characteristics of a glaucoma patient.

[0123] In an embodiment of the present application, step 13 includes the following substeps.

[0124] Substep 134: evaluate the eye movement data in the detection time to obtain task indicators.

[0125] When a key event of the eye movement experiment occurs, such as the line of sight contacting the correct instance object or the line of sight contacting the incorrect instance object, the event script of Unity will be triggered immediately to record the event and the time point when the event occurs.

[0126] Task indicators refer to the goals or standards set in the eye movement detection task, which are used to evaluate the performance of participants when performing tasks. The task indicators provided in this step include but are not limited to accuracy, effective recognition duration, recognition duration, fixation target duration, reaction time, and reaction correct time.

[0127] This substep includes the following substeps.

[0128] Substep 1341: evaluate the eye movement data in the fixation detection time to obtain fixation task indicators.

[0129] This substep includes the following task indicators.

[0130] Fixation accuracy: the number of successfully eliminated instance objects divided by the total number of instance objects.

[0131] Gaze effective recognition duration: the total duration of fixation on the target object within the experimental duration (e.g. 30s).

[0132] Gaze recognition duration: the total duration of fixation on the target object within the experimental duration (e.g. 30s). This includes the accumulated duration of fixation on the target object but without maintaining fixation.

[0133] Gaze reaction time: the duration from the appearance of the target object to the movement of the gaze onto the target object.

[0134] Sub-step 1342: evaluate the eye movement data of the tracking detection time to obtain tracking task indicators.

[0135] This sub-step includes the following task indicators.

[0136] Tracking accuracy: the number of successfully eliminated target objects divided by the total number of target objects presented.

[0137] Tracking effective recognition duration: the total duration of smooth pursuit of the target object to elimination within the experimental duration (e.g. 30s).

[0138] Tracking recognition duration: the total duration of smooth pursuit of the target object within the experimental duration (e.g. 30s). This includes the accumulated duration of smooth pursuit to the target object but without maintaining fixation.

[0139] Tracking reaction time: the duration from the appearance of the target object to the movement of the gaze onto the target object.

[0140] Sub-step 1343: evaluate the eye movement data of the matching detection time to obtain matching task indicators.

[0141] This sub-step includes the following task indicators.

[0142] Matching accuracy: the number of successfully matched target objects divided by the total number of target objects presented.

[0143] Matching gaze target duration: the total duration of fixation on the target object (including correct and incorrect target objects).

[0144] Matching reaction correct time: the duration from the appearance of the target object to the movement of the gaze onto the correct target object.

[0145] Matching reaction time: the duration from the appearance of the target object to the movement of the gaze onto the target object (including correct and incorrect target objects).

[0146] Sub-step 135: compare the task indicators obtained in the above steps with the data of the baseline experiment to determine whether the subject meets the characteristics of glaucoma patients.

[0147] The task indicators obtained in sub-step 134 can quantitatively evaluate the eye movement characteristics of the glaucoma patients under different illumination conditions and different tasks, such as the number of fixations, fixation time, etc., analyze and illustrate the influence of the visual field defects of the glaucoma patients on the eye movement behavior, and provide a basis for subsequent personalized visual training of the glaucoma patients and improvement of the life quality.

[0148] The fixation task indicators can analyze the following characteristics.

[0149] The eye movement event indicators of the 10° fixation task are shown in Table 2. It can be known from the results that the normal control group is better than the glaucoma group in the performance of the correct rate, effective recognition duration, recognition duration and reaction time, and the correct rate and effective recognition duration have a significant difference at the level of p<0.05, which indicates that the efficiency and accuracy of the eye health subjects in completing the task are higher; and the monocular severe group is relatively worst in the performance of the four indicators, which may be due to the fact that the difference between the two eyes of the monocular severe patients has a more serious influence on the visual search task.

[0150] Table 2 Eye movement event indicators: 10° fixation task

[0151]

[0152] The results of the 24° fixation task are shown in Table 3. The correct rate and effective recognition duration have a significant difference between the groups at p<0.05, and the recognition duration has a significant difference at p<0.05; between the experimental group and the control group as a whole, the effective recognition duration and the recognition duration have a statistical difference at p<0.05, and the reaction time has a significant difference at p<0.05. And compared with the 10° fixation task, the 24° fixation task shows stronger difference between the groups, which is because the visual field defects of the glaucoma patients generally start from the periphery and gradually damage inward, and the loss of the peripheral visual field of the patients is more serious than that of the central visual field, and the 24° fixation range is larger than the 10° range, and thus the difference between the groups due to the degree of visual field defects is more obvious. From the result values, the control group performs best in the four scores, and the performance gradually deteriorates with the increase of the degree of visual field defects, and the binocular severe group performs worst, which is in line with the expectation.

[0153] Table 3 Eye movement event indicators: 24° fixation task

[0154]

[0155] 10° smooth pursuit task results are shown in Table 4. Among them, the binocular severe group is the worst in accuracy, effective recognition time, recognition time and reaction time. After statistical calculation, the binocular severe group and the rest of the group have statistical significance in the above two indicators (p<0.05). But there is no statistical difference between the groups and the experimental group and the control group as a whole, which shows that the binocular severe group has poor ability to maintain continuous tracking of visual line.

[0156] Table 4: Eye movement event indicators: 10° smooth pursuit task

[0157]

[0158] 24° smooth pursuit task results are shown in Table 5. The results are consistent with the 10° smooth pursuit, with the binocular severe group performing the worst and the normal control group performing the best. And because the 24° experimental range is larger, the performance gap between groups is more obvious. Specifically, for the same experimental task, there is no group difference in the 10° range, and when the range is increased to 24°, the effective recognition time has a group difference at the p<0.1 level, and the recognition time and reaction time of the experimental group and the control group as a whole have differences at the p<0.1 and p<0.05 levels, respectively.

[0159] Table 5: Eye movement event indicators: 24° smooth pursuit task

[0160]

[0161] In addition to the above overall data comparison, the inventors also analyzed the experimental results of the groups in black and white backgrounds. For the 24° fixation task, the reaction time of the binocular moderate and below group in the white background was significantly faster than that in the black background, with a statistical difference at the p<0.1 level; for the 10° smooth pursuit task, the accuracy of the binocular severe group and the binocular moderate and below group in the white background was higher (p<0.1, p<0.05), and the reaction time of the three glaucoma experimental groups in the white background was faster (all p<0.1). There is no statistical difference between the control group in black and white backgrounds in each experimental task.

[0162] From the overall results of the black and white backgrounds, glaucoma patients tend to perform better in the white background with higher brightness, while this difference in background brightness is not observed in normal subjects. The white background can provide more ambient light and contrast, and sufficient light can help glaucoma patients perceive the target of the visual task. Patients are more sensitive to light differences, while normal subjects have no visual field defects, and the change in brightness during the experiment will not affect the results.

[0163] The evaluation results of the visual matching task are listed in Table 6. From the results, the double-eye severe group has the lowest correct rate, the double-eye moderate group has relatively poor performance in correct fixation target duration, fixation target duration, and reaction correct time, and the single-eye severe group has the slowest reaction time. There is no statistical difference in the five evaluation indicators among the groups. Compared with the visual search task, the visual matching task is basically located in the central field of view, and the target size is 2.5 times that of the search task target, so the discrimination for glaucoma patients is not obvious, which indicates that the visual matching function in the central visual field of the patient is not significantly affected by the visual field defect. The visual function in this part is relatively complete. Statistical analysis shows that there is no statistical difference in the five test indicators under black and white backgrounds.

[0164] Table 6 Eye movement event indicators: visual matching task

[0165]

[0166] In an embodiment of the present application, the glaucoma-normal person classification model is obtained by the following steps.

[0167] Step 21: Diagnose the eye health of the subject through medical examination.

[0168] The purpose of this step is to diagnose whether the subject is a healthy person or a glaucoma patient, so as to divide the training set of the obtained detection data.

[0169] Step 22: The subject wears a VR device and calibrates the gaze point.

[0170] This step is the same as step 11.

[0171] Step 23: Enter the pre-detection time, display the image of the detection scene on the display module, collect the eye movement and head movement data of the subject when observing the image, and calibrate.

[0172] This step is the same as step 12.

[0173] Step 24: Enter the detection time, display the image of the detection scene on the display module, and collect the eye movement and head movement data of the subject when observing the image.

[0174] This step is the same as step 13. It includes the following sub-steps.

[0175] Sub-step 241: Enter the gaze detection time, display the gaze detection content on the display module, and the subject gazes at the detection content. The gaze detection content is an example object in a static state within a circular visual field range of 10° and 24°.

[0176] This step is the same as step 131.

[0177] Sub-step 242: Enter the tracking detection time, display the tracking detection content on the display module, and the subject gazes at the tracking detection content. The tracking detection content is an example object in a moving state within a 10° and 24° circular field of view.

[0178] This step is the same as step 132.

[0179] Sub-step 243: Enter the matching detection time, display the matching detection content on the display module, and the subject gazes at the matching detection content. The matching detection content is a plurality of example objects in a stationary state within the field of view.

[0180] This step is the same as step 133.

[0181] Step 25: Merge the eye movement data and head movement data based on the time point.

[0182] This step is the same as step 14.

[0183] Step 26: Obtain data of a sufficient number of diagnosed subjects to form a data set, and divide the training set and the test set of the data set.

[0184] For example, two kinds of data of the same subject under the same scene are processed in time sequence, and a merged csv file is obtained. The overall sampling frequency is 60 Hz, which is down-sampled to 20 Hz, and 25 rows are selected. Every 25 rows of time sequence data is a group, and 9 columns of eye movement data and 7 columns of head movement data are respectively flattened into a row, a total of 25x(9+7)=400 as the feature quantity of the deep learning data set. The category of the file corresponding to the subject is used as the label, 0 represents a normal subject, and 1 represents a glaucoma patient. If the number of rows of the merged file is not a multiple of 25, the last group is not included in the data set. The training set and the test set are divided according to a ratio of about 3:1. Finally, after merging and splitting, data enhancement, and data division, a training set with a size of 9148x400 and a test set with a size of 3208x400 are obtained. Among them, the proportion of category 0 in the training set is 44%, the proportion of category 1 is 30%, and the total proportion in the data set is 74%. The proportion of category 0 in the test set is 15%, the proportion of category 1 is 11%, and the total proportion in the data set is 26%. The distribution ratio of the training set, the test set, and the two categories of labels is shown in the table as shown in the table. Figure 3

[0185] Step 27: Based on the 1DCNN model and the training set, a glaucoma-normal person classification model is trained and established, and the test set is used to evaluate the model.

[0186] ​Deep learning models, especially models like Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs), have highly complex internal structures. They learn complex patterns and features from large amounts of input data, but these features are not always intuitive or easily interpretable. Although the models perform well and can make accurate predictions, it is not possible to directly extract explicit human-understandable features from the models. Therefore, this step includes the following sub-steps.

[0187] Sub-step 271: The network model adds LSTM or Attention layers based on the 1D CNN model and the training set to build 1D-CNN, LSTM+1D CNN, 1D CNN+Attention, and LSTM+1D CNN+Attention models for training.

[0188] The head eye movement module input channel number is 16, and the output channel number is 128. The convolution kernel list is [3, 4, 5], which means that 3x1, 4x1, and 5x1 convolution kernels are used to perform convolution operations on the input one-dimensional data to capture local features of different sizes; and a max pooling layer is applied with a pooling kernel size of 2.

[0189] The LSTM network is added before the 1D CNN module. The data first passes through the respective LSTM layer, with an input dimension of 16 and an output of 128; the CNN receives data from the LSTM for encoding, with an input and output of 128, and the convolution kernel and pooling layer settings remain unchanged.

[0190] The attention mechanism is added after the output of the CNN, with one attention head for each attention layer. Two attention layers automatically weight the input data during forward propagation and output the weighted features, which helps the model capture key information in the input data. The output features are input to the fully connected network. The fully connected network consists of two linear layers and an activation layer, with a random dropout (Dropout) operation between the two linear layers to enhance network performance and reduce the likelihood of overfitting. The number of classification categories is set to 2, and the final output is the corresponding classification result.

[0191] During training, five-fold cross-validation is used, with a total of 100 epochs, a learning rate of 0.001, a batch size of 32, and Cross Entropy Loss as the loss function.

[0192] Sub-step 272: Obtain model evaluation indicators based on the test set.

[0193] For binary classification models, the common evaluation method is to classify the total samples according to the actual label and the predicted label into true positive, false positive, true negative and false negative samples. Specifically, true positive (True Positive) indicates that the model successfully detects glaucoma patients, true negative (True Negative) indicates that the model correctly identifies normal people, false positive (False Positive) indicates misdiagnosis of normal people, and false negative (False Negative) indicates missed diagnosis of glaucoma patients.

[0194] The evaluation indexes such as accuracy, precision, recall, F1 score, ROC curve and AUC, confusion matrix, etc. can be used to evaluate the deep learning model from the aspects of training process, test results, etc. to determine the performance and implementation effect of the model.

[0195] Accuracy (Accuracy) refers to the proportion of correctly classified samples in the total samples, and the formula is described as formula (1).

[0196] (1)

[0197] Precision (Precision) refers to the proportion of correctly predicted positive samples in all predicted positive samples, and the formula is described as formula (2).

[0198] (2)

[0199] Recall (Recall) refers to the proportion of samples predicted as positive to the actual positive samples, and the formula is described as formula (3).

[0200] (3)

[0201] F1 score (F1 score) is the harmonic mean of precision and recall, which describes the comprehensive performance of the model, and the formula is described as formula (4).

[0202] (4)

[0203] The ROC curve describes the relationship between true positive rate and false positive rate, and AUC is the area under the ROC curve. The closer the AUC is to 1, the better the model performance; the confusion matrix shows the classification results of the model through the number of true positive, false positive, true negative and false negative samples.

[0204] Sub-step 273: Select the best model according to the evaluation index.

[0205] Get the loss function curve. After 100 iterations, get the loss function curve to describe the loss function trend of the training set and the test set. For example,Figure 4A As shown in the figure, the 1DCNN+Attention model has the fastest decline speed in the first 10 epochs of the training process. After about 10 epochs, the loss values of the four models all decrease to 0.1, and the convergence speed is very fast. At this time, the convergence speed of the 1DCNN+Attention model is the slowest. After 70 epochs, the loss values of the models all converge to about 0.001. As shown in the figure, the loss values of the four models all decrease to about 0.001 after 70 epochs. Figure 4B As shown in the figure, the loss function values of the four models finally decrease to below 0.4 on the test set. The 1DCNN model has the best effect and the change is relatively stable, followed by the LSTM+1DCNN+Attention, LSTM+1DCNN, and 1DCNN+Attention models.

[0206] Evaluation index results. The evaluation indexes of the deep learning model, accuracy, precision, recall, F1 score, and AUC results are shown in Table 7. From the size of the five indexes, the 1DCNN and Attention layer have a greater improvement on the model. The LSTM-1DCNN-Attention combined model reaches an accuracy of 86.6%, which has the best accuracy, precision, recall, and F1 score among all models, and has better comprehensive performance. The AUC value of the 1DCNN-Attention model is the highest. As shown in the figure, the results of the binary classification confusion matrix can be seen that the designed deep learning model has the ability to distinguish between glaucoma patients and normal people. Figures 5A-5D

[0207] Table 7 Performance evaluation indexes of deep learning model

[0208]

[0209] Figures 5A-5D ​The following information is shown by the binary classification confusion matrix of the 1D CNN, 1D CNN-Attention, LSTM-1D CNN, and LSTM-1D CNN-Attention models: ① True Positive: the number of samples correctly predicted by the model as positive (actual positive, predicted positive) (lower right block); ② False Positive: the number of samples incorrectly predicted by the model as positive (actual negative, predicted positive), also called "Type I error" or "false positive" (upper right block); ③ False Negative: the number of samples incorrectly predicted by the model as negative (actual positive, predicted negative), also called "Type II error" or "false negative" (lower left block); ④ True Negative: the number of samples correctly predicted by the model as negative (actual negative, predicted negative) (upper left block). Through this matrix, various evaluation indicators such as accuracy, precision, recall, F1 value, etc. can be calculated to comprehensively measure the performance of the model.

[0210] The baseline experimental results of the embodiments of the present application show that the severe glaucoma group performs worse in the visual search task, and the results of 24° visual field are more obvious than those of 10° visual field. In the visual matching task, glaucoma patients have lower evaluation than normal people.

[0211] Second embodiment

[0212] Based on the first embodiment described above, the second embodiment of the present application provides a screening device for distinguishing early glaucoma patients, which is used to implement the screening method provided in the above embodiments. The screening device includes a VR device and a control device, which communicate with each other in a wireless or wired manner.

[0213] The VR device includes a display module, a head movement sensor, an eye movement sensor, an audio module, and a communication module. The display module includes a head-mounted display for displaying images to the user. The head movement sensor is used to track and record the movement and rotation data of the user's head. The eye movement sensor is used to track and record the gaze direction data of the pupil. The audio module is used to play audio prompts. The communication module is used to realize data transmission.

[0214] The control device includes a computing module and a storage module. The computing module includes a calculator, such as a CPU, GPU, NPU, and a special-purpose chip (such as a mobile SoC), which is used to provide data calculation capability. The storage module includes a memory and a storage, which is used to provide data storage capability. The computing module reads the computer program in the storage module, which is used to execute the screening method for distinguishing early glaucoma patients described above.

[0215] It should be noted that the above embodiments are only illustrative. The technical solutions of various embodiments can be combined, and all are within the protection scope of the present application.

[0216] The terms "first", "second", are only used for descriptive purposes and should not be construed as indicating or implying relative importance or an indicated number of technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0217] The screening method and screening device for distinguishing early glaucoma patients provided by the present application are described in detail above. Any obvious modification made to the present application by a person skilled in the art without departing from the essential content of the present application will constitute an infringement of the patent right of the present application and will bear the corresponding legal responsibility.

Claims

1. A screening device for distinguishing early-stage glaucoma patients, comprising a VR device and a control device, characterized in that: The VR device includes a display module, a head motion sensor, an eye movement sensor, an audio module, and a communication module; the display module is used to display images; the head motion sensor is used to acquire head motion data; the eye movement sensor is used to acquire eye movement data; the audio module is used to play audio prompts; and the communication module is used to implement data transmission. The control device includes a computing module and a storage module. The computing module reads a computer program from the storage module to execute the following screening method: Step 11: Subjects wear VR devices and calibrate their gaze points; Step 12: Enter the pre-detection time, display the image of the pre-detection scene on the display module of the VR device, collect the eye movement data and head movement data of the subject when observing the image, and perform calibration; Step 13: Enter the detection time, display the image of the detection scene on the display module, and collect the eye movement data and head movement data of the subject when observing the image; wherein, the target instance object in the detection scene is regarded as the child object of the VR camera, and the world coordinates of the child object relative to the world coordinates of the parent object and the coordinate vector of the child object in the coordinate system of the parent object are used to eliminate the interference of head movement on eye movement detection; Step 14: Based on the time point, merge the eye movement data and the head movement data, including the following sub-steps: Sub-step 141: Read the eye-tracking data and the head-tracking data into a DataFrame tabular data structure; Sub-step 142: Store the timestamp difference between the eye-tracking data and the head-tracking data; store the smallest timestamp difference found, with an initial value set to infinity; record the index of the last merged row in the head-tracking data, with an initial value of 0; Sub-step 143: Starting from line 0, traverse each line of the eye-tracking data until line m, where m is a positive integer; Sub-step 144: Traverse each row of the header data starting from the row specified by the index, up to the i-th row, where i is a positive integer; Sub-step 145: Calculate the timestamp difference of the current row; if the difference is less than the minimum timestamp difference of the current record, update the minimum timestamp difference; otherwise, merge the m-th row of the eye-tracking data and the i-th row of the head-tracking data; reset the timestamp difference and the minimum timestamp difference, and update the index to the index i of the current row; Sub-step 146: Exit sub-step 144 and continue with sub-step 143 until the eye-tracking data has been traversed. Sub-step 147: Save the merged time series data; Step 15: Substitute the merged data obtained in the above steps into the glaucoma-normal person classification model for evaluation to determine whether the subject meets the characteristics of a glaucoma patient.

2. The screening device as described in claim 1, characterized in that... Step 13 includes the following sub-steps: Sub-step 131: Enter the gaze detection time, the gaze detection content is displayed on the display module, the gaze detection content of the subject; the gaze detection content is an instance object in a stationary state within a circular field of view of 10° and 24° in a white and black panoramic background; Sub-step 132: Enter the tracking detection time, the tracking detection content is displayed on the display module, and the subject gazes at the tracking detection content; the tracking detection content is an instance object in motion within a 10° and 24° circular field of view in a white and black panoramic background; Sub-step 133: Enter the matching detection time, the matching detection content is displayed on the display module, and the subject gazes at the matching detection content; the matching detection content is multiple instance objects in a stationary state within the field of vision in a white and black panoramic background.

3. The screening device as described in claim 2, characterized in that: The panoramic backgrounds for steps 12 and 13 are white and black, with the white and black backgrounds alternating during the step transitions.

4. The screening device as described in claim 2, characterized in that... Step 13 includes the following sub-steps: Sub-step 134: Evaluate eye-tracking data during the detection time to obtain task metrics; Sub-step 135: Compare the task indicators with the baseline experimental data to determine whether the subject meets the characteristics of a glaucoma patient.

5. The screening device as described in claim 1, characterized in that... The method for obtaining the glaucoma-normal person classification model includes the following steps: Step 21: Confirm the subject's eye health status through medical examination; Step 22: The subject wears the VR device and calibrates the gaze point; Step 23: Enter the pre-detection time, display the image of the detection scene on the display module, collect the eye movement and head movement data of the subject when observing the image, and perform calibration; Step 24: Enter the detection time, display the image of the detection scene on the display module, and collect the eye movement and head movement data of the subject when observing the image; Step 25: Merge the eye movement data and the head movement data based on the time point; Step 26: Obtain sufficient data from confirmed subjects to form a dataset, and divide the dataset into a training set and a test set; Step 27: Based on the 1DCNN model and the training set, train and establish the glaucoma-normal person classification model, and evaluate the model using the test set.

6. The screening device as described in claim 5, characterized in that... Step 27 includes the following sub-steps: Sub-step 271: Based on the 1DCNN model and training set, add LSTM or Attention layers to the CNN to build 1D-CNN model, LSTM+1DCNN model, 1DCNN+Attention model and LSTM+1DCNN+Attention model and carry out training. Sub-step 272: Obtain model evaluation metrics based on the test set; Sub-step 273: Select the model with the best performance based on the evaluation indicators.

Images