A three-dimensional space-based eye movement evaluation method and device

Abstract

The application provides a three-dimensional space-based eye movement evaluation method and device, which is applied to an eye movement evaluation system. The method comprises the following steps: selecting a corresponding target test question in a preset test question group based on the symptoms of a patient, determining a corresponding target three-dimensional scene based on the target test question, responding to the patient staring at a target model in the target three-dimensional scene according to a prompt, moving the target model in the three-dimensional space based on the target test question, determining a target model setting coordinate of the target model, emitting a preset infrared light point array to the eyeball of the patient, collecting target video information of the eyeball of the patient according to a preset target camera, processing the infrared video information and the color video information based on a preset video algorithm to obtain a target convergence point coordinate of the patient, and calculating an eye movement evaluation score of the patient according to the target model setting coordinate and the target convergence point coordinate.

Description

Technical Field

[0001] This application relates to the field of medical technology, and more specifically, to a three-dimensional eye-tracking assessment method and device. Background Technology

[0002] Neurodegenerative diseases are among the most prevalent illnesses in the elderly population. Abnormal brain aging leading to impaired motor function can cause numerous inconveniences and even dangerous situations, such as difficulties with daily activities, cognitive dysfunction, and memory decline. Alzheimer's disease, a typical neurodegenerative disease, is characterized by cognitive impairment, manifested as bradykinesia and a decline in judgment, calculation, and memory abilities. Therefore, early detection and treatment are crucial.

[0003] Currently, patients' eye movements are mainly assessed using traditional cognitive scales and eye movement assessments.

[0004] However, traditional scale assessments are mostly subjective judgments, which are greatly influenced by the testers. Therefore, they can only be used as auxiliary assessment methods and quantitative assessment is needed as the decisive basis. Traditional eye movement assessments are mostly assessments of the angle of rotation of a single eye. They often show the angle of the left and right eyes separately, or simply show the fixation point on a plane. It is difficult to conduct quantitative assessment in a more refined three-dimensional space, resulting in general measurement dimensions and test accuracy. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a three-dimensional eye-tracking assessment method and device, which can acquire color video information and infrared video information corresponding to the patient staring at the target model in the target three-dimensional scene through a color camera and an infrared camera, respectively. The coordinates of the patient's target convergence point are obtained based on the infrared video information and color video information, and the patient's eye-tracking assessment score is calculated based on the target model's set coordinates and the target convergence point coordinates. This realizes three-dimensional eye-tracking assessment of the patient in three-dimensional space, reduces the influence of the tester, and improves the measurement dimension and test accuracy.

[0006] In a first aspect, embodiments of this application provide a three-dimensional space-based eye-tracking assessment method, applied to an eye-tracking assessment system, the method comprising:

[0007] Based on the patient's symptoms, a corresponding target test question is selected from a preset test question group, and a corresponding target 3D scene is determined based on the target test question; wherein, the target 3D scene includes at least a prompt and a target model;

[0008] In response to the patient staring at the target model in the target 3D scene according to the prompt, the target model is moved in 3D space based on the target test question, and the target model's target model set coordinates are determined;

[0009] A preset infrared light array is emitted towards the patient's eyeball, and target video information of the patient's eyeball is acquired by a preset target camera; wherein, the target camera includes a color camera and an infrared camera; the target video information includes color video information acquired by the color camera and infrared video information acquired by the infrared camera;

[0010] The infrared video information and the color video information are processed based on a preset video algorithm to obtain the target convergence point coordinates of the patient, and the eye movement assessment score of the patient is calculated based on the target model set coordinates and the target convergence point coordinates.

[0011] In one possible implementation, the step of processing the infrared video information and the color video information based on a preset video algorithm to obtain the target convergence point coordinates of the patient includes:

[0012] The infrared video information is processed based on a preset video algorithm to obtain the coordinates of the first convergence point, and the color video information is processed to obtain the coordinates of the second convergence point.

[0013] The first convergence point coordinates are corrected based on the second convergence point coordinates to obtain the target convergence point coordinates of the patient.

[0014] In one possible implementation, the step of correcting the first convergence point coordinates based on the second convergence point coordinates to obtain the target convergence point coordinates of the patient includes:

[0015] Obtain the target confidence ratio; wherein the target confidence ratio is a widely validated confidence ratio;

[0016] The first and second convergence point coordinates are weighted and averaged based on the target confidence ratio to obtain the corresponding target convergence point coordinates.

[0017] In one possible implementation, the infrared video information is processed based on a preset video algorithm to obtain the coordinates of the first convergence point:

[0018] The infrared video information is processed based on a preset video algorithm to obtain the planar image features of the patient's eyeball, and the patient's first eyeball movement angle is obtained based on the planar image features and the infrared light dot matrix.

[0019] The coordinates of the patient's first convergence point are calculated based on the first eye movement angle and the preset target calculation formula.

[0020] In one possible implementation, before calculating the patient's first convergence point coordinates based on the first eye movement angle and a preset target calculation formula, the method further includes:

[0021] For the target 3D scene, a target point is placed at the target calibration position in the target 3D scene, and the second eye movement angle and the distance to the target point are determined; wherein, the distance to the target point is the distance from the target point to the patient's eyeball;

[0022] The target camera is zeroed based on the target point.

[0023] In one possible implementation, the target calculation formula includes an interpupillary distance calculation formula and a convergence point coordinate formula. The step of calculating the patient's first convergence point coordinates based on the first eye movement angle and the preset target calculation formula includes:

[0024] The patient's target pupillary distance is calculated based on the second eye movement angle, the target point distance, and the pupillary distance calculation formula.

[0025] The coordinates of the patient's first convergence point are calculated based on the first eye movement angle, the target interpupillary distance, and the convergence point coordinate formula.

[0026] In one possible implementation, calculating the patient's eye movement assessment score based on the target model set coordinates and the target convergence point coordinates includes:

[0027] The corresponding calculation frequency is determined based on the image acquisition frequency of the target camera;

[0028] Within each calculation frequency, the real-time error of that calculation frequency is calculated based on the target model set coordinates and the target convergence point coordinates, and the total average error is obtained based on all the real-time errors;

[0029] For the patient, the total test duration is determined, and the patient's eye movement assessment score is calculated based on the total test duration and the total average error.

[0030] Secondly, embodiments of this application also provide a three-dimensional space-based eye-tracking assessment device, which applies an eye-tracking assessment system. The device includes:

[0031] The selection module is used to select a corresponding target test question from a preset test question group based on the patient's symptoms, and to determine a corresponding target 3D scene based on the target test question; wherein, the target 3D scene includes at least a prompt and a target model;

[0032] The determination module is used to respond to the patient staring at the target model in the target 3D scene according to the prompt, move the target model in 3D space based on the target test question, and determine the target model set coordinates of the target model;

[0033] The acquisition module is used to emit a preset infrared light array to the patient's eyeball and acquire target video information of the patient's eyeball according to a preset target camera; wherein, the target camera includes a color camera and an infrared camera; the target video information includes color video information acquired by the color camera and infrared video information acquired by the infrared camera;

[0034] The calculation module is used to process the infrared video information and the color video information based on a preset video algorithm to obtain the target convergence point coordinates of the patient, and to calculate the patient's eye movement assessment score according to the target model set coordinates and the target convergence point coordinates.

[0035] In one possible implementation, the computing module is specifically used for:

[0036] The infrared video information is processed based on a preset video algorithm to obtain the coordinates of the first convergence point, and the color video information is processed to obtain the coordinates of the second convergence point.

[0037] The first convergence point coordinates are corrected based on the second convergence point coordinates to obtain the target convergence point coordinates of the patient.

[0038] In one possible implementation, the computing module is specifically used for:

[0039] Obtain the target confidence ratio; wherein the target confidence ratio is a widely validated confidence ratio;

[0040] The first and second convergence point coordinates are weighted and averaged based on the target confidence ratio to obtain the corresponding target convergence point coordinates.

[0041] In one possible implementation, the computing module is specifically used for:

[0042] The infrared video information is processed based on a preset video algorithm to obtain the planar image features of the patient's eyeball, and the patient's first eyeball movement angle is obtained based on the planar image features and the infrared light dot matrix.

[0043] The coordinates of the patient's first convergence point are calculated based on the first eye movement angle and the preset target calculation formula.

[0044] In one possible implementation, the computing module is specifically used for:

[0045] Before calculating the patient's first convergence point coordinates based on the first eye movement angle and a preset target calculation formula, a target point is placed at the target calibration position in the target three-dimensional scene to determine the patient's second eye movement angle and the target point distance; wherein, the target point distance is the distance from the target point to the patient's eyeball;

[0046] The target camera is zeroed based on the target point.

[0047] In one possible implementation, the target calculation formula includes an interpupillary distance calculation formula and a convergence point coordinate formula. The calculation module is specifically used to: calculate the patient's target interpupillary distance based on the second eye movement angle, the target point distance, and the interpupillary distance calculation formula.

[0048] The coordinates of the patient's first convergence point are calculated based on the first eye movement angle, the target interpupillary distance, and the convergence point coordinate formula.

[0049] In one possible implementation, the computing module is specifically used for:

[0050] The corresponding calculation frequency is determined based on the image acquisition frequency of the target camera;

[0051] Within each calculation frequency, the real-time error of that calculation frequency is calculated based on the target model set coordinates and the target convergence point coordinates, and the total average error is obtained based on all the real-time errors;

[0052] For the patient, the total test duration is determined, and the patient's eye movement assessment score is calculated based on the total test duration and the total average error.

[0053] Thirdly, embodiments of this application provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the eye-tracking assessment method based on three-dimensional space as described in any of the first aspects.

[0054] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the eye-tracking assessment method based on three-dimensional space as described in any of the first aspects.

[0055] This application provides a three-dimensional eye-tracking assessment method and apparatus. Based on the patient's symptoms, a corresponding target test question is selected from a preset test question set. A corresponding target three-dimensional scene is determined based on the target test question. Responding to the patient's gaze at a target model in the target three-dimensional scene according to prompts, the target model is moved in three-dimensional space based on the target test question. The target model's set coordinates are determined. A preset infrared light array is emitted to the patient's eyeball. Target video information from the patient's eyeball is acquired using a preset target camera. The infrared and color video information are processed using a preset video algorithm to obtain the patient's target convergence point coordinates. The patient's eye-tracking assessment score is calculated based on the target model's set coordinates and the target convergence point coordinates. In this application, color and infrared cameras are used to acquire color and infrared video information corresponding to the patient's gaze at the target model in the target three-dimensional scene. The patient's target convergence point coordinates are obtained based on the infrared and color video information. The patient's eye-tracking assessment score is calculated based on the target model's set coordinates and the target convergence point coordinates. This achieves three-dimensional eye-tracking assessment of the patient in three-dimensional space, reducing the influence of the tester and improving measurement dimensionality and test accuracy.

[0056] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0057] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart of a three-dimensional space-based eye-tracking assessment method according to an embodiment of this application;

[0059] Figure 2 This is a schematic diagram of an eye-tracking assessment system;

[0060] Figure 3 This is a flowchart of a three-dimensional space-based eye-tracking assessment method according to another embodiment of this application;

[0061] Figure 4 This is a flowchart of a three-dimensional space-based eye-tracking assessment method according to another embodiment of this application;

[0062] Figure 5 This is a flowchart of a three-dimensional space-based eye-tracking assessment method according to another embodiment of this application;

[0063] Figure 6 This is a schematic diagram of the structure of a three-dimensional space-based eye-tracking assessment device according to an embodiment of this application;

[0064] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0066] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0067] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0068] Given that neurodegenerative diseases are among the most prevalent illnesses in the elderly population, the impaired motor function caused by abnormal brain aging can lead to numerous inconveniences and even dangerous situations, such as difficulties in daily activities, cognitive dysfunction, and memory decline. Alzheimer's disease, a typical neurodegenerative disease, is characterized by cognitive impairment, manifested primarily as bradykinesia and a decline in judgment, calculation, and memory abilities. Therefore, early detection and treatment of this condition are crucial.

[0069] Currently, patients' eye movements are mainly assessed using traditional cognitive scales and eye movement assessments. However, traditional scale assessments are largely subjective and heavily influenced by the test-taker, thus serving only as supplementary assessment methods. Quantitative assessment is needed as the definitive basis. Furthermore, traditional eye movement assessments often only evaluate the angle of rotation of one eye, typically showing the angles of the left and right eyes individually, or simply displaying the fixation point on a plane. This makes it difficult to perform quantitative assessments in a more refined three-dimensional space, resulting in limited measurement dimensionality and testing accuracy.

[0070] To address this issue, this application provides a three-dimensional eye-tracking assessment method and apparatus. It uses a color camera and an infrared camera to collect color and infrared video information corresponding to the patient's gaze at a target model within a target three-dimensional scene. The coordinates of the patient's target convergence point are obtained based on the infrared and color video information. The patient's eye-tracking assessment score is then calculated based on the target model's set coordinates and the target convergence point coordinates. This achieves three-dimensional eye-tracking assessment of the patient within a three-dimensional space, reducing the influence of the testing personnel and improving measurement dimensionality and testing accuracy.

[0071] Figure 1 This is a flowchart of a three-dimensional space-based eye-tracking assessment method according to an embodiment of this application. This three-dimensional space-based eye-tracking assessment method is applied to an eye-tracking assessment system, such as... Figure 1 As shown, the eye-tracking assessment method based on three-dimensional space in this application embodiment may specifically include the following steps:

[0072] S101, based on the patient's symptoms, select the corresponding target test question from the preset test question group, and determine the corresponding target three-dimensional scene based on the target test question.

[0073] In this embodiment, the target 3D scene includes at least a prompt and a target model. The prompt is used to guide the patient through the test, and the target model is a model set in the 3D scene, such as... Figure 2 As shown, the point above the two eyeballs is the target model. The patient can be a patient with neurodegenerative disease. The preset test question group is a combination of multiple questions that are pre-set to conduct eye movement assessment tests on the patient. For example, the target model moves left and right, moves up and down, and jumps. The target test question is the question that conducts eye movement assessment tests on the patient. For example, the target model is moved left and right. The target test question is selected from the preset test question group according to the patient's symptoms, and the target three-dimensional scene is determined based on the target test question for subsequent processing.

[0074] S102, in response to the patient staring at the target model in the target 3D scene according to the prompt, moves the target model in 3D space based on the target test questions, and determines the target model's target model set coordinates.

[0075] In this embodiment, the target model's set coordinates are the coordinates of the target model. After selecting a target test item based on the patient's symptoms, an eye-tracking test is initiated. Once the test begins, the patient observes a target 3D scene and focuses on the target model within that scene according to prompts. The target model moves in 3D space according to the target test item. For example, if the target test item is "jumping," the target model jumps in 3D space. The coordinates of the target model at this time are the target model's set coordinates (Xc, Yc). C Z C ), for subsequent processing.

[0076] Optional, such as Figure 2 As shown, the patient observes the target 3D scene and the target model in the target 3D scene through a VR headset. S103, a preset infrared light array is emitted to the patient's eyeball, and the target video information of the patient's eyeball is collected according to the preset target camera.

[0077] In this embodiment, the infrared light dot matrix refers to a pre-set dot matrix infrared light source. The patient's eyeballs refer to both eyes. The target camera includes a color camera and an infrared camera. The target video information includes color video information captured by the color camera and infrared video information captured by the infrared camera. The infrared light dot matrix is ​​emitted to the patient's eyeballs, and the target video information from the patient's eyeballs is collected by the target camera for subsequent processing. It should be noted that the target video information is collected at the beginning of the eye-tracking test. Specifically, when the infrared light dot matrix is ​​emitted to the patient's eyeballs, the color camera and the infrared camera work simultaneously to collect the corresponding video information.

[0078] S104 processes infrared and color video information based on a preset video algorithm to obtain the patient's target convergence point coordinates, and calculates the patient's eye movement assessment score based on the target model set coordinates and the target convergence point coordinates.

[0079] In this embodiment, the video algorithm is a pre-set algorithm for processing video captured by a camera, such as an algorithm in the OpenCV video algorithm library. The target convergence point coordinates (represented by (X, Y, Z) in this application) are the convergence point coordinates corresponding to the patient's eyeballs when observing the target model. The eye movement assessment score is the score of the patient's eye movement test assessment. The target convergence point coordinates of the patient are obtained by processing infrared video information and color video information according to the video algorithm, and the patient's eye movement assessment score is calculated according to the target model set coordinates and the target convergence point coordinates, thereby completing the quantitative assessment of the patient's eye movement.

[0080] This enables quantitative assessment of patients' eye-movement cognition based on three-dimensional space.

[0081] The eye-tracking assessment method based on three-dimensional space provided in this application selects a corresponding target test question from a preset test question group based on the patient's symptoms, and determines a corresponding target three-dimensional scene based on the target test question. Responding to the patient staring at a target model in the target three-dimensional scene according to prompts, the target model is moved in three-dimensional space based on the target test question, and the target model's set coordinates are determined. A preset infrared light array is emitted to the patient's eyeball. Target video information of the patient's eyeball is acquired using a preset target camera. The infrared video information and color video information are processed based on a preset video algorithm to obtain the patient's target convergence point coordinates. The patient's eye-tracking assessment score is calculated based on the target model's set coordinates and the target convergence point coordinates. This three-dimensional eye-tracking assessment method, through color and infrared cameras respectively acquiring color and infrared video information corresponding to the patient staring at the target model in the target three-dimensional scene, obtains the patient's target convergence point coordinates based on the infrared and color video information, and calculates the patient's eye-tracking assessment score based on the target model's set coordinates and the target convergence point coordinates, realizing three-dimensional eye-tracking assessment of the patient in three-dimensional space, reducing the influence of the tester, and improving measurement dimensionality and test accuracy.

[0082] Furthermore, such as Figure 3 As shown, step S104 in the above embodiment, "processing infrared video information and color video information based on a preset video algorithm to obtain the target convergence point coordinates of the patient," may specifically include the following steps:

[0083] S301, based on a preset video algorithm, process infrared video information to obtain the coordinates of the first convergence point, and process color video information to obtain the coordinates of the second convergence point.

[0084] In this embodiment of the application, the first convergence point coordinates are the convergence point coordinates (X1, Y1, Z1) corresponding to the infrared video information, and the second convergence point coordinates are the convergence point coordinates (X2, Y2, Z2) corresponding to the color video information. The first convergence point coordinates (X1, Y1, Z1) and the second convergence point coordinates (X2, Y2, Z2) are obtained by processing the infrared video information and the color video information respectively based on a preset video algorithm, so as to perform subsequent processing.

[0085] S302, the coordinates of the first convergence point are corrected based on the coordinates of the second convergence point to obtain the coordinates of the patient's target convergence point.

[0086] In this embodiment of the application, the first convergence point coordinates in step S301 are corrected based on the second convergence point coordinates obtained in step S301 to obtain the target convergence point coordinates of the patient. That is, the first convergence point coordinates (X1, Y1, Z1) are corrected by the second convergence point coordinates (X2, Y2, Z2) to obtain the target convergence point coordinates (X, Y, Z).

[0087] Understandably, both the video algorithms for processing infrared video information and color video information are based on OpenCV's open-source algorithms. However, they belong to two different branches of algorithms. The video algorithm for infrared video information emphasizes the input of specified coordinates, while the video algorithm for color video information emphasizes the input image. Obviously, coordinates are more accurate than images. Therefore, the first convergence point coordinates obtained from infrared video information are more accurate than the second convergence point coordinates obtained from color video information. However, color video information has a large amount of image information and can be used as an auxiliary tool. That is, the first convergence point coordinates can be corrected by using the second convergence point coordinates. In this way, the target convergence point coordinates obtained are more accurate than the first convergence point coordinates.

[0088] It should be noted that this application does not impose too many restrictions on the specific method of correcting the coordinates of the first convergence point based on the coordinates of the second convergence point, and can be set according to the actual situation.

[0089] As one possible implementation, a target confidence ratio is obtained; based on the target confidence ratio, a weighted average is calculated on the coordinates of the first and second convergence points to obtain the corresponding target convergence point coordinates. Here, the target confidence ratio is a widely validated confidence ratio. For example, with a target confidence ratio of 7:3, the coordinates of the second convergence point are (X2, Y2, Z2), and the coordinates of the first convergence point are (X1, Y1, Z1). Based on the confidence ratio of 7:3, a weighted average is calculated on (X2, Y2, Z2) and (X1, Y1, Z1) to obtain the target convergence point coordinates (X, Y, Z), where X = 0.7*X1 + 0.3*X2, Y = 0.7*Y2 + 0.3*Y2, and Z = 0.7*Z1 + 0.3*Z2. The corrected target convergence point coordinates are then calculated accordingly.

[0090] Therefore, the video information transmitted by the color camera and the infrared camera was processed separately, and the processing results were combined. That is, the coordinates of the first convergence point of the infrared camera were corrected based on the coordinates of the second convergence point of the color camera, realizing dual-channel verification of the convergence point coordinates and increasing the test accuracy.

[0091] Furthermore, such as Figure 4 As shown, step S301 in the above embodiment, "processing infrared video information based on a preset video algorithm to obtain the coordinates of the first convergence point," may specifically include the following steps:

[0092] S401 processes infrared video information based on a preset video algorithm, obtains planar image features of the patient's eyeball, and obtains the patient's first eyeball movement angle based on the planar image features and the infrared light dot matrix.

[0093] In this embodiment of the application, the planar image features are the two-dimensional planar image features of the patient's eyeball, and the first eyeball movement angle includes α1, β1, and γ1, such as... Figure 2 As shown, α1 and γ1 are the angles of the distance (connection line) between the target model and one eyeball (pupil) relative to the X-axis and Z-axis, respectively, and β1 is the angle of the distance (connection line) between the target model and the other eyeball (pupil) relative to the X-axis. It can be understood that the Z-axis angle of both eyeballs is γ1. The first eyeball movement angle characterizes the angle of the patient's real-time changing eyeball movement. By processing the black and white image in the infrared video information according to the video algorithm, the planar image features of the patient's eyes can be obtained. Based on the planar image features and the infrared light dot matrix emitted in the above embodiment, the patient's first eyeball movement angle is obtained for subsequent processing.

[0094] As one possible implementation, the target protruding part of the patient's eyeball is determined based on planar image features and an infrared dot matrix, and the corresponding coordinates of the black pupil edge point are obtained based on the target protruding part. Based on video algorithms, the coordinates of the black pupil edge point and multiple coordinates in the infrared dot matrix are processed to determine the three-dimensional motion characteristics of the patient's eyeball within a preset time period, and the first eyeball motion angle is obtained based on these three-dimensional motion characteristics. For example, by combining the infrared dot matrix, the protruding part of the eyeball can be highlighted. The obtained black pupil edge point coordinates, combined with multiple coordinates in the infrared dot matrix, can be processed using the OpenCV video algorithm library to output three-dimensional modeling machine motion information of the protruding eyeball. Based on the changes in the pupil edge coordinates and the coordinates of the infrared dot matrix over a period of time (i.e., the changes in multiple coordinates in the infrared dot matrix across different time dimensions), the three-dimensional motion characteristics of the two eyeballs during this period can be reconstructed, i.e., the real-time changing eyeball motion angles, which are the first eyeball motion angles α1, β1, and γ1.

[0095] S402 calculates the patient's first convergence point coordinates based on the first eye movement angle and the preset target calculation formula.

[0096] In this embodiment of the application, the target calculation formula is a pre-set calculation formula, which includes the pupillary distance calculation formula and the convergence point coordinate formula. The first convergence point coordinates of the patient are calculated based on the first eye movement angle obtained in step S401 and the target calculation formula for subsequent processing.

[0097] The formula for calculating pupillary distance is as follows:

[0098] K=L*cosα*sinγ+L'*cosβ*sinγ

[0099] It should be noted that before calculating the patient's first convergence point coordinates based on the first eye movement angle and a preset target calculation formula, a target point is placed at the target calibration position within the target 3D scene to determine the patient's second eye movement angle and the distance to the target point; the target camera is then zeroed based on this target point. The target calibration position is the location corresponding to the center of the target 3D scene at infinity, and the target point distance is the distance from the target point to the patient's eyeball, i.e., the distance from the target point to the patient's eyeball before the eye movement test. For example, as shown... Figure 2 The L and L' shown represent the second eye movement angle, which is the same as the first eye movement angle, including α2, β2, and γ2. The second eye movement angle characterizes the angle of the patient's real-time changing eye movements. It can be understood that before the eye-tracking test, a target point is placed at the target calibration position in the target 3D scene, and the color camera and infrared camera are zeroed based on this target point. Since the coordinates of an infinitely distant point are 0, i.e., the target point coordinates are 0, the patient's second eye movement angles α2, β2, and γ2 are all known at this time, and the distance to the target point is also known, thus determining the patient's second eye movement angles and the distance to the target point. The video information recognized by the infrared and color cameras is defined as the original 0-point state of the eyeball. Since the eyeball is basically stationary and relatively stable in this state, the patient's pupillary distance K can be calculated in this state using the aforementioned pupillary distance calculation formula.

[0100] It should be noted that this application does not impose too many restrictions on the specific method of calculating the coordinates of the first convergence point based on the first eye movement angle and the target calculation formula, and can be set according to the actual situation.

[0101] As one possible implementation, the patient's target pupillary distance is calculated based on the second eye movement angle, target point distance, and pupillary distance calculation formula; the patient's first convergence point coordinates are calculated based on the first eye movement angle, target pupillary distance, and convergence point coordinate formula.

[0102] The formula for the convergence point coordinates is as follows:

[0103] X=L*sinα*sinγ=L'sinβ*sinγ

[0104] Y=L*cosα*sinγ=K-L'cosβ*sinγ

[0105] Z = L * cosγ = L'cosγ

[0106] For example, by substituting the patient's second eye movement angles α2, β2, and γ2, as well as the target point distance, into the above pupillary distance calculation formula, the patient's target pupillary distance K can be calculated.

[0107] Optionally, the target model distance for the patient during the eye movement test can be calculated based on the target interpupillary distance and the first eye movement angle. The target model distance is the distance from the target model to the patient's eyeball. For example, as shown in the figure, given the target interpupillary distance K and the first eye movement angles α1, β1, and γ1, the target model distances L and L' for the patient during the eye movement test can be obtained. Based on the aforementioned convergence point coordinate formula, X1, Y1, and Z1 can be calculated, thus obtaining the patient's first convergence point coordinates (X1, Y1, Z1).

[0108] Similarly, the patient's second convergence point coordinates (X2, Y2, Z2) are obtained based on the above formula, which will not be described in detail in this application.

[0109] Furthermore, such as Figure 5 As shown, step S104 in the above embodiment, "calculating the patient's eye movement assessment score based on the target model's set coordinates and the target convergence point coordinates," may specifically include the following steps:

[0110] S501 determines the corresponding calculation frequency based on the image acquisition frequency of the target camera.

[0111] In this embodiment of the application, each calculation frequency corresponds to one calculation. The image acquisition frequency is the frequency at which the target camera acquires images, for example, 60Hz. The corresponding calculation frequency is determined according to the image acquisition frequency of the target camera. For example, when the image acquisition frequency is 60Hz, the corresponding calculation frequency is 1 / 60s, that is, one calculation is performed every 1 / 60s.

[0112] S502, within each calculation frequency, calculate the real-time error of that calculation frequency based on the target model set coordinates and the target convergence point coordinates, and obtain the total average error based on all real-time errors.

[0113] In this embodiment of the application, within each calculation frequency obtained in step S501 above, coordinates (X) are set according to the target model. c Y c Z c The real-time error ΔM for this calculation frequency is calculated using the coordinates (X, Y, Z) of the target convergence point and the target convergence point. The total average error M is then obtained based on all the real-time errors ΔM. The formula for calculating the real-time error ΔM is as follows:

[0114]

[0115] It should be noted that the total average error M is the quotient of the sum of all real-time errors ΔM divided by the sum of the total calculation frequencies. For example, if the calculation frequency is 1 / 60s and 60 calculations are performed, that is, a total of 1 second of calculations, then the total average error M is obtained by adding the 60 real-time errors ΔM together and dividing by 1 second.

[0116] S503, for patients, determines the total test duration and calculates the patient's eye movement assessment score based on the total test duration and the total average error.

[0117] In this embodiment, the total test duration is the total time recorded for the patient throughout the entire eye movement test. The patient's eye movement assessment score S is calculated based on the total test duration T and the total average error M obtained in the above embodiments. Optionally, the eye movement assessment score S can be calculated using S = M / T.

[0118] To clearly describe the three-dimensional space-based eye-tracking assessment method of this application, the following will be combined with the above... Figure 2 Describe it.

[0119] For example, such as Figure 2 As shown, the eye-tracking assessment system includes a data acquisition device and a control device. The data acquisition device consists of a VR headset and a near-eye infrared eye-tracking module, which includes a color camera and an infrared camera. The control device is a general-purpose computer. Connecting the acquisition and control devices and opening the control software serves as the platform for overall process control, data calculation, and subsequent result display. To conduct an eye-tracking test, the patient selects a test question group based on their symptoms on the control device. Once the test begins, the patient can see a target 3D scene in the VR headset. This scene includes a background model, prompts, and the target model. The patient focuses on the target model according to the prompts, and the target model moves in 3D space according to the test requirements. At this time, the target model's coordinates are (Xc, Yc, Zc). The acquisition device collects eye-tracking information (video information) at the start of the test and emits an infrared image array onto the eyeball. Simultaneously, the color camera and infrared camera work together to transmit the video information to the control device. The control unit calculates the average error, total average error, and records the total test duration. It further calculates the patient's eye movement assessment score. Specifically, it calculates the real-time error based on the difference between the calculated target convergence point coordinates (X, Y, Z) and the target point set coordinates (Xc, Yc, Zc). The result is calculated once every 1 / 60s, ΔM = √((xX...)). c ) 2 +(yY c ) 2 +(zZ c ) 2 Finally, the total statistical average error M is calculated, and the total test duration T is recorded. After the eye movement test is completed, the eye movement assessment score S = M / T is calculated based on the statistical error value.

[0120] Figure 6 This is a flowchart of a three-dimensional space-based eye-tracking assessment device according to an embodiment of this application, which is applied to a mobile terminal, such as... Figure 6As shown, the three-dimensional space-based eye-tracking assessment device 600 of this application embodiment may specifically include:

[0121] The selection module 601 is used to select the corresponding target test question from the preset test question group based on the patient's symptoms, and to determine the corresponding target three-dimensional scene based on the target test question; wherein, the target three-dimensional scene includes at least a prompt and a target model.

[0122] The determination module 602 is used to respond to the patient staring at the target model in the target 3D scene according to the prompt, move the target model in 3D space based on the target test questions, and determine the target model set coordinates.

[0123] The acquisition module 603 is used to emit a preset infrared light array to the patient's eyeball and acquire target video information of the patient's eyeball according to a preset target camera; wherein, the target camera includes a color camera and an infrared camera; the target video information includes color video information acquired by the color camera and infrared video information acquired by the infrared camera.

[0124] The calculation module 604 is used to process infrared video information and color video information based on a preset video algorithm to obtain the target convergence point coordinates of the patient, and to calculate the patient's eye movement assessment score according to the target model set coordinates and the target convergence point coordinates.

[0125] In one possible implementation, the computing module is specifically used for:

[0126] The first convergence point coordinates are obtained by processing infrared video information based on a preset video algorithm, and the second convergence point coordinates are obtained by processing color video information.

[0127] The coordinates of the first convergence point are corrected based on the coordinates of the second convergence point to obtain the coordinates of the patient's target convergence point.

[0128] In one possible implementation, the computing module is specifically used for:

[0129] Obtain the target confidence ratio; where the target confidence ratio is the confidence ratio of broad validation.

[0130] The target convergence point coordinates are obtained by weighted averaging the coordinates of the first and second convergence points based on the target confidence ratio.

[0131] In one possible implementation, the computing module is specifically used for:

[0132] The infrared video information is processed based on a preset video algorithm to obtain the planar image features of the patient's eyeballs, and the patient's first eyeball movement angle is obtained based on the planar image features and the infrared light dot matrix.

[0133] The patient's first convergence point coordinates are calculated based on the first eye movement angle and a preset target calculation formula.

[0134] In one possible implementation, the computing module is specifically used for:

[0135] Before calculating the patient's first convergence point coordinates based on the first eye movement angle and the preset target calculation formula, a target point is placed at the target calibration position in the target three-dimensional scene to determine the patient's second eye movement angle and the target point distance; wherein, the target point distance is the distance from the target point to the patient's eyeball;

[0136] Zero-calibrate the target camera based on the target point.

[0137] In one possible implementation, the target calculation formula includes a pupillary distance calculation formula and a convergence point coordinate formula. The calculation module is specifically used to: calculate the patient's target pupillary distance based on the second eye movement angle, the target point distance, and the pupillary distance calculation formula.

[0138] The patient's first convergence point coordinates are calculated based on the first eye movement angle, target pupillary distance, and convergence point coordinate formula.

[0139] In one possible implementation, the computing module is specifically used for:

[0140] The corresponding calculation frequency is determined based on the image acquisition frequency of the target camera;

[0141] Within each calculation frequency, the real-time error of that calculation frequency is calculated based on the target model set coordinates and the target convergence point coordinates, and the total average error is obtained based on all the real-time errors;

[0142] For each patient, the total test duration is determined, and the patient's eye movement assessment score is calculated based on the total test duration and the total average error.

[0143] The three-dimensional eye-tracking assessment device provided in this application selects a corresponding target test question from a preset test question group based on the patient's symptoms, and determines a corresponding target three-dimensional scene based on the target test question. Responding to the patient's gaze at the target model in the target three-dimensional scene according to prompts, the device moves the target model in three-dimensional space based on the target test question, determines the target model's set coordinates, emits a preset infrared light array to the patient's eyeball, collects target video information from the patient's eyeball using a preset target camera, processes the infrared and color video information based on a preset video algorithm to obtain the patient's target convergence point coordinates, and calculates the patient's eye-tracking assessment score based on the target model's set coordinates and the target convergence point coordinates. This three-dimensional eye-tracking assessment device, through a color camera and an infrared camera, respectively collects color and infrared video information corresponding to the patient's gaze at the target model in the target three-dimensional scene, obtains the patient's target convergence point coordinates based on the infrared and color video information, and calculates the patient's eye-tracking assessment score based on the target model's set coordinates and the target convergence point coordinates. This achieves three-dimensional eye-tracking assessment of the patient in three-dimensional space, reduces the influence of the tester, and improves measurement dimensionality and test accuracy.

[0144] like Figure 7 As shown in the embodiment of this application, an electronic device 700 includes a processor 701, a memory 702, and a bus. The memory 702 stores machine-readable instructions executable by the processor 701. When the electronic device is running, the processor 701 communicates with the memory 702 via the bus, and the processor 701 executes the machine-readable instructions to perform the steps of the eye-tracking assessment method based on three-dimensional space as described above.

[0145] Specifically, the memory 702 and processor 701 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 701 runs the computer program stored in the memory 702, it can execute the above-mentioned eye-tracking assessment method based on three-dimensional space.

[0146] Corresponding to the above-described eye-tracking assessment method based on three-dimensional space, this application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described eye-tracking assessment method based on three-dimensional space.

[0147] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0148] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0150] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the deployment methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0151] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A three-dimensional space-based eye-tracking assessment method, characterized in that, Applied to an eye-tracking assessment system; the method includes: Based on the patient's symptoms, a corresponding target test question is selected from a preset test question group, and a corresponding target 3D scene is determined based on the target test question; wherein, the target 3D scene includes at least a prompt and a target model; In response to the patient staring at the target model in the target 3D scene according to the prompt, the target model is moved in 3D space based on the target test question, and the target model's target model set coordinates are determined; A preset infrared light array is emitted to the patient's eyeball, and target video information of the patient's eyeball is acquired by a preset target camera; wherein, the target camera includes a color camera and an infrared camera; the target video information includes color video information acquired by the color camera and infrared video information acquired by the infrared camera; The infrared video information and the color video information are processed based on a preset video algorithm to obtain the target convergence point coordinates of the patient, and the eye movement assessment score of the patient is calculated according to the target model set coordinates and the target convergence point coordinates. The process of obtaining the target convergence point coordinates based on the preset video algorithm includes processing the infrared video information to obtain first convergence point coordinates and processing the color video information to obtain second convergence point coordinates; correcting the first convergence point coordinates based on the second convergence point coordinates to obtain the patient's target convergence point coordinates; and correcting the first convergence point coordinates based on the second convergence point coordinates to obtain the patient's target convergence point coordinates, which includes: obtaining a target confidence ratio; wherein the target confidence ratio is a widely validated confidence ratio; and calculating a weighted average of the first and second convergence point coordinates based on the target confidence ratio to obtain the corresponding target convergence point coordinates.

2. The method according to claim 1, characterized in that, The infrared video information is processed using a preset video algorithm to obtain the coordinates of the first convergence point: The infrared video information is processed based on a preset video algorithm to obtain the planar image features of the patient's eyeball, and the patient's first eyeball movement angle is obtained based on the planar image features and the infrared light dot matrix. The coordinates of the patient's first convergence point are calculated based on the first eye movement angle and the preset target calculation formula.

3. The evaluation method according to claim 2, characterized in that, Before calculating the patient's first convergence point coordinates based on the first eye movement angle and a preset target calculation formula, the method further includes: For the target 3D scene, a target point is placed at the target calibration position in the target 3D scene to determine the patient's second eye movement angle and the distance to the target point; wherein, the distance to the target point is the distance from the target point to the patient's eyeball; The target camera is zeroed based on the target point.

4. The evaluation method according to claim 3, characterized in that, The target calculation formula includes an interpupillary distance calculation formula and a convergence point coordinate formula. The calculation of the patient's first convergence point coordinates based on the first eye movement angle and the preset target calculation formula includes: The patient's target pupillary distance is calculated based on the second eye movement angle, the target point distance, and the pupillary distance calculation formula. The coordinates of the patient's first convergence point are calculated based on the first eye movement angle, the target interpupillary distance, and the convergence point coordinate formula.

5. The evaluation method according to claim 1, characterized in that, The calculation of the patient's eye movement assessment score based on the target model coordinates and the target convergence point coordinates includes: The corresponding calculation frequency is determined based on the image acquisition frequency of the target camera; Within each calculation frequency, the real-time error of that calculation frequency is calculated based on the target model set coordinates and the target convergence point coordinates, and the total average error is obtained based on all the real-time errors; For the patient, the total test duration is determined, and the patient's eye movement assessment score is calculated based on the total test duration and the total average error.

6. A three-dimensional space-based eye-tracking assessment device, characterized in that, The device, used in an eye-tracking assessment system, includes: The selection module is used to select a corresponding target test question from a preset test question group based on the patient's symptoms, and to determine a corresponding target 3D scene based on the target test question; wherein, the target 3D scene includes at least a prompt and a target model; The determination module is used to respond to the patient staring at the target model in the target 3D scene according to the prompt, move the target model in 3D space based on the target test question, and determine the target model set coordinates of the target model; The acquisition module is used to emit a preset infrared light array to the patient's eyeball and acquire target video information of the patient's eyeball according to a preset target camera; wherein, the target camera includes a color camera and an infrared camera; the target video information includes color video information acquired by the color camera and infrared video information acquired by the infrared camera; The calculation module is used to process the infrared video information and the color video information based on a preset video algorithm to obtain the target convergence point coordinates of the patient, and to calculate the patient's eye movement assessment score according to the target model set coordinates and the target convergence point coordinates; wherein, the calculation module is specifically used to: process the infrared video information based on the preset video algorithm to obtain the first convergence point coordinates, and process the color video information to obtain the second convergence point coordinates; correct the first convergence point coordinates based on the second convergence point coordinates to obtain the patient's target convergence point coordinates; the calculation module is also used to obtain the target confidence ratio; wherein, the target confidence ratio is a widely validated confidence ratio; and calculate the weighted average of the first convergence point coordinates and the second convergence point coordinates based on the target confidence ratio to obtain the corresponding target convergence point coordinates.

7. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the eye-tracking assessment method based on three-dimensional space as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the eye-tracking assessment method based on three-dimensional space as described in any one of claims 1 to 5.

Images