Eye-tracking testing methods, devices, systems, electronic equipment, and readable storage media

Abstract

This application discloses an eye-tracking testing method, apparatus, system, electronic device, and readable storage medium, relating to the field of computer vision technology. Specific implementations include: a computer device sending prompts to a head-mounted VR device, indicating the test content during the eye-tracking test to the target subject; then acquiring the 3D coordinates of the target subject's eye movements during the eye-tracking test collected by the head-mounted VR device; converting the 3D coordinates to 2D coordinates and displaying them on the computer device; allowing real-time observation and understanding of the target subject's eye movements during the eye-tracking test from the computer device. This solves the problem in related technologies where the patient's eye movements cannot be seen when using a head-mounted VR device for eye-tracking testing, thus providing data support for eye-tracking test guidance and improving the effectiveness and accuracy of eye-tracking tests.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to the field of computer vision technology, and in particular to eye-tracking testing methods, apparatus, systems, electronic devices, and readable storage media. Background Technology

[0002] Eye movement testing involves using medical equipment to measure the movement trajectory of the eyeballs. It is one of the main methods for diagnosing strabismus, especially paralytic strabismus, and can also reflect the mental state of the examinee to some extent.

[0003] In routine eye movement (EMT) examinations, head-mounted VR (Virtual Reality) devices are used to acquire patients' eye movement information. The VR device guides patients through pre-set scenarios to perform eye movements, thus completing the EMT test. However, in actual testing, because the VR environment is closed when the patient uses the device, the doctor cannot see the patient's eye movements. When patients cannot understand the instructions, the doctor struggles to accurately guide them, leading to test failure and an inability to correctly detect the patient's eye movements. Summary of the Invention

[0004] To address the issue that external observers cannot see a patient's eye movements when using a head-mounted VR device to perform eye movement tests, an eye movement test method, device, and system are provided.

[0005] According to the first aspect, an eye-tracking testing system is provided, comprising: a head-mounted VR device for acquiring the 3D coordinates of the eye movement of a target object during the eye-tracking test; and a computer device, communicatively connected to the head-mounted VR device, for converting the 3D coordinates of the eye movement into 2D coordinates, so as to display the eye movement of the target object in a set size on the display interface of the computer device.

[0006] According to the second aspect, an eye-tracking test method is provided, comprising: sending a prompting message to a head-mounted VR device, wherein the prompting message is used to prompt the target object with the test content during the eye-tracking test; acquiring the 3D coordinates of the target object's eye movement during the eye-tracking test collected by the head-mounted VR device; and converting the 3D coordinates of the eye movement into 2D coordinates so as to display the target object's eye movement in a display interface at a set size.

[0007] According to a third aspect, an eye-tracking testing device is provided, comprising: a sending unit configured to send prompting information to a head-mounted VR device, wherein the prompting information is used to prompt the target object with the test content during the eye-tracking test; an acquisition unit configured to acquire the eye-tracking 3D coordinates of the target object collected by the head-mounted VR device during the eye-tracking test; and a display unit configured to convert the eye-tracking 3D coordinates into 2D coordinates to display the eye-tracking status of the target object in a display interface at a set size.

[0008] According to a fourth aspect, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any embodiment of the eye-tracking testing method.

[0009] According to a fifth aspect, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method as described in any embodiment of the eye-tracking testing method.

[0010] Using the technical solution of this application, the computer device sends prompts to the head-mounted VR device, prompting the target object with the test content during the eye-tracking test. Then, it acquires the 3D coordinates of the target object's eyes during the eye-tracking test collected by the head-mounted VR device. After converting the 3D coordinates of the eye-tracking to 2D coordinates, it displays them on the computer device. The target object's eye movement can be observed and understood in real time from the computer device. This solves the problem in related technologies where the patient's eye movement cannot be seen when using a head-mounted VR device to conduct eye-tracking tests on the target object. This provides data support for eye-tracking test guidance and helps improve the effectiveness and accuracy of eye-tracking tests. Attached Figure Description

[0011] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0012] Figure 1 This is an exemplary system architecture diagram in which some embodiments of this application can be applied;

[0013] Figure 2 This is a flowchart of one embodiment of the eye-tracking testing method according to this application;

[0014] Figure 3 This is a schematic diagram of one embodiment of the eye-tracking testing device according to this application;

[0015] Figure 4 This is a block diagram of an electronic device used to implement the eye-tracking testing method of the embodiments of this application. Detailed Implementation

[0016] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] Figure 1 An exemplary eye-tracking testing system 100 is shown, in which embodiments of the eye-tracking testing method or eye-tracking testing apparatus of this application can be applied.

[0019] like Figure 1 As shown, the system architecture 100 may include a head-mounted VR device 101, a computer device 102, and a communication device 103. The head-mounted VR device 101 acquires the 3D coordinates of the patient's eye movements during the eye-tracking test; the computer device converts the 3D eye-tracking coordinates into 2D coordinates for display; the communication device 103 serves as a medium for providing a communication link between the head-mounted VR device 101 and the computer device 102, and may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0020] Computer device 102 can be various electronic devices with a display screen, including but not limited to smartphones, tablets, e-book readers, laptops, and desktop computers.

[0021] It should be noted that the eye-tracking testing method provided in this application embodiment can be executed by computer device 102, such as performing the following steps: filtering out invalid 3D coordinates from the eye-tracking 3D coordinates based on the eye movement speed to obtain valid 3D coordinates; mapping the valid 3D coordinates onto multiple two-dimensional planes for display, wherein each two-dimensional plane is used to display the eye movement from one visual angle, and any two two-dimensional planes display different visual angles. Accordingly, the eye-tracking testing device can be set in computer device 102.

[0022] Continue to refer to Figure 2 The flowchart 200 of one embodiment of the eye-tracking testing method according to this application is shown. In this embodiment, the eye-tracking testing method runs on an execution entity (e.g., Figure 1 (The computer equipment shown). This eye-tracking test method includes the following steps:

[0023] Step 201: The computer device sends prompts to the head-mounted VR device. The prompts are used to inform the target (i.e., the person who needs to perform the eye movement test, such as a patient) about the test content or guidance during the eye movement test, such as: the actions to be performed, the standard posture of the actions, and the description of the actions.

[0024] Step 202: The computer device acquires the 3D coordinates of the target object's eye movement during the eye-tracking test, collected by the head-mounted VR device.

[0025] For example, computer devices and head-mounted VR devices can be connected via communication cables or wireless methods (such as Bluetooth) to obtain the eye-tracking 3D coordinates of the target object's eye position in real time. The eye position refers to the current gaze point of the eye. For example, when the eye is looking at a certain place in space, the 3D coordinates of the gaze point are (0,0,0). Then, the 3D coordinates of the eye position are called the eye position position (0,0,0).

[0026] Step 203: The computer device converts the eye-tracking 3D coordinates into 2D coordinates so that the eye-tracking of the target object can be displayed in the display interface at a set size. The set size can be the default display size in the display interface or a display size dynamically set by the user during the viewing process.

[0027] In some optional implementations of this embodiment, in order to improve the accuracy, effectiveness, and comprehensiveness of the computer device displaying eye movement, it is proposed to filter the eye movement 3D coordinates and display them through multiple two-dimensional planes with different visual angles. For example, invalid 3D coordinates can be filtered out from the eye movement 3D coordinates according to the speed of eye movement to obtain valid 3D coordinates. The valid 3D coordinates are then mapped to multiple two-dimensional planes (such as 2, 3, 4, etc.) for display. Each two-dimensional plane is used to display the eye movement from one visual angle, and any two two-dimensional planes display different visual angles.

[0028] In some optional implementations of this embodiment, invalid 3D coordinates refer to abnormal data caused by situations such as squinting or blinking. The inventors, through analysis of these scenarios, discovered that the data in these scenarios exhibits a characteristic of excessively large or small speed changes. Therefore, they proposed a method to accurately and effectively filter out invalid 3D coordinates using the characteristic of eye movement speed. The specific implementation is as follows:

[0029] 1) Obtain the first eye-tracking 3D coordinate in the eye-tracking 3D coordinate system. Second eye movement 3D coordinates Among them, the first eye movement 3D coordinates Second eye movement 3D coordinates These are the 3D coordinates of the eye movements for each of the two eyes.

[0030] 2) Set the first eye movement 3D coordinates Second eye movement 3D coordinates Substitute the values ​​into the following formula to calculate the eye movement parameter θ, which represents the speed of eye movement:

[0031]

[0032] 3) When the eye-tracking parameter θ is less than the first threshold (indicating that the velocity change is too small; the value of the first threshold can be determined according to the specific situation, such as 0.2) or greater than the second threshold (indicating that the velocity change is too large; the value of the second threshold can be determined according to the specific situation, such as 1), the first eye-tracking 3D coordinates are... Second eye movement 3D coordinates Invalid 3D coordinates are filtered out from eye-tracking 3D coordinates. The second threshold is greater than the first threshold. The larger the value of the eye-tracking parameter θ, the faster the eye moves.

[0033] 4) When the eye-tracking parameter θ is greater than or equal to the first threshold and less than or equal to the second threshold, the first eye-tracking 3D coordinates are... Second eye movement 3D coordinates Retained as valid 3D coordinates.

[0034] In some optional implementations of this embodiment, since the display size of the computer screen varies, to meet different display requirements under different coordinate viewing conditions, it is proposed that the 3D data collected by the VR device be converted into 2D coordinates first, and then the 2D coordinates be converted into display sizes according to the size of the display screen adjusted according to different display requirements. For example, after completing coordinate filtering, the coordinates can be converted as follows:

[0035] Using the width w, height h, and original eye size s (which can be represented by a radius) of the two-dimensional plane (i.e., the display interface), and the eye scaling factor m, the first eye-tracking 3D coordinates, which serve as effective 3D coordinates, are determined. Second eye movement 3D coordinates Perform mapping:

[0036]

[0037]

[0038] Obtain the mapped x-axis coordinates x C y-axis coordinates after mapping C and the mapped z-axis coordinate z C Then, the first 2D coordinate (x) C yC The second 2D coordinates (x, y) are mapped onto the first two-dimensional plane for display, and the second 2D coordinates (x, y) are mapped onto the first two-dimensional plane for display. C , z C The third 2D coordinate (z) is mapped onto the second two-dimensional plane for display, and the third 2D coordinate (z) is mapped onto the second two-dimensional plane for display. C u C It is then mapped onto a third two-dimensional plane for display.

[0039] In practice, the above formula is applied normally in the two-dimensional plane XY (i.e., the first two-dimensional plane). C The calculation formula and y C The calculation formula; using x in the two-dimensional plane XZ (i.e., the second two-dimensional plane). C The calculation formula and z C The calculation formula is used to calculate z. C When, the "w|h" part takes the "h" part. Use the "-" sign in the middle; use y in the two-dimensional plane YZ (i.e., the third two-dimensional plane). C The calculation formula and z C The calculation formula is used to calculate z. C When, the "w|h" part takes "w". The value of m is 1 when no scaling is applied.

[0040] In some optional implementations of this embodiment, in order to more clearly and prominently display the eye movement in certain special situations when displaying the target object's eyeball using a two-dimensional plane, it is proposed to display a reference eyeball image by opening other two-dimensional planes, so as to accurately understand the movement of the eyeball.

[0041] Firstly, if the target subject has previously undergone eye-tracking testing, the trajectory diagram of the historical eye-tracking test, the number of instantaneous movements, the speed of excessively fast instantaneous movements, and the speed of excessively slow instantaneous movements are displayed in the historical two-dimensional plane (which is newly launched relative to the existing two-dimensional plane), so as to understand the eye-tracking situation in the historical eye-tracking test based on the data displayed in the historical two-dimensional plane.

[0042] Secondly, when the eye movement speed is large or small, i.e., when the eye moves instantaneously, a new two-dimensional plane can be started for display, which makes it easier to understand the eye movement before and after the instantaneous movement: when the eye movement parameter θ is greater than or equal to the first threshold (e.g., 0.2) and less than the third threshold (e.g., 0.3) (i.e., the instantaneous movement speed is too slow), or when the eye movement parameter θ is greater than the fourth threshold (e.g., 0.8) and less than or equal to the second threshold (e.g., 1.0) (i.e., the instantaneous movement speed is too fast), it is determined that the target object's eye has moved instantaneously. That is, multiple two-dimensional planes are restarted to display the eye movement, so as to show the comparison results before and after the instantaneous movement by comparing the original multiple two-dimensional planes with the restarted multiple two-dimensional planes. Among them, the third threshold is greater than the first threshold and less than the fourth threshold, and the fourth threshold is less than the second threshold.

[0043] In practical implementation, existing technologies using head-mounted VR devices for eye-tracking testing suffer from limitations because the dynamic eye movements of the target subject cannot be observed and understood in real time. This hinders accurate guidance for the subject and leads to numerous incorrect or invalid tests. However, the eye-tracking testing solution proposed in this application involves a computer sending prompts to the head-mounted VR device, indicating the test content to the target subject. The computer then acquires the 3D coordinates of the target subject's eyes during the eye-tracking test, collected by the head-mounted VR device. These 3D coordinates are converted to 2D coordinates and displayed on the computer device, allowing for real-time observation and understanding of the target subject's eye movements. This solves the problem of not being able to see the patient's eye movements when using head-mounted VR devices for eye-tracking testing in related technologies. Furthermore, it provides data for guiding eye-tracking testing, improving its effectiveness and accuracy.

[0044] In practice, the above-mentioned eye-tracking testing method can be applied to any scenario that requires eye-tracking testing. For example, it can be used in medical eye-tracking testing scenarios, as well as in routine eye-tracking testing scenarios such as eye inspection and selection tests.

[0045] In specific implementation, the above-mentioned eye movement testing method, also known as the adaptive matrix transformation algorithm, is described in detail below with reference to specific implementation methods. This method solves the problem of being able to observe and understand the patient's eye movement during the eye movement testing process, and thus can correctly guide the eye movement testing.

[0046] 1) Establish a data connection between the head-mounted VR device and the computer so that the two ends can interact with each other. For example, you can use a data cable to connect the head-mounted VR device and the computer, or you can establish a connection through Bluetooth, local area network and other communication methods.

[0047] 2) The patient wears a VR headset, and the person guiding the test can operate it on a computer to display the test content on the VR headset and perform eye-tracking tests on the patient (i.e., the target subject mentioned above). After operation on the computer, the test content to be performed is displayed synchronously on the VR headset, and the patient performs eye-tracking tests according to the test content.

[0048] 3) Once the test begins, the VR device captures the patient's eye movement 3D coordinate data in real time and transmits the eye movement data to the computer.

[0049] 4) After receiving the eye-tracking data transmitted from the VR device, the computer opens three two-dimensional plane layers, for example, displaying the XY plane (i.e., the first two-dimensional plane), XZ plane (i.e., the second two-dimensional plane), and ZY plane (i.e., the third two-dimensional plane) of the 3D coordinate image respectively. In the two-dimensional plane layers, dots are used to represent the eye's focal point or gaze point in the VR device (this dot can be a circular model representing the eye's focal point or gaze point in the plane; the specific form of the identifier representing the eye's focal point or gaze point in the two-dimensional plane layer is not limited, it can be a dot, or it can be an image of an eye, etc.).

[0050] 4.1) The coordinate data transmitted from the VR device to the computer is the eye-tracking 3D coordinates in the three-dimensional coordinate system with the origin at the center of the virtual screen displayed on the VR device. It is necessary to first filter out invalid coordinate data collected by the VR device (such as abnormal data caused by blinking or squinting). The filtering process requires calculating the eye movement speed first. Eye movement data with excessively high or low speeds are judged as invalid coordinate data.

[0051]

[0052] When the θ value is greater than 1 (i.e., the second threshold mentioned above) or less than 0.2 (i.e., the first threshold mentioned above), it is judged as invalid coordinate data.

[0053] 4.2) Then convert the three-dimensional coordinate system after screening and washing into a Cartesian coordinate system with the origin in the upper left corner of the display interface that can be recognized by the computer.

[0054] Extract the X and Y axis values, X and Z axis values, and Z and Y axis values ​​from the three-dimensional coordinate system, as well as the width w and height h of the computer plane, and convert them into x-axis values ​​in the planar direct coordinate system with the top left corner as the origin using an adaptive matrix transformation algorithm. C With y C Axis value, x C With z C Axis value, z C With y CThe X and Y axis values ​​are converted and displayed on the first two-dimensional plane on the computer, the X and Z axis values ​​are displayed on the second two-dimensional plane on the computer, and the Z and Y axis values ​​are displayed on the third two-dimensional plane on the computer, so that the patient's eye movement can be observed and understood in real time through two-dimensional planes.

[0055] 4.3) When the person supervising the test observes deviations in the patient's eye movement position during the test on the computer, they can zoom in or out by a factor of m to display the coordinate dots on the three two-dimensional planes. During zooming, the size s and position of the dots are recalculated according to the zoom ratio. After zooming, the size of the dots displayed on the planes will be readjusted according to the zoom ratio, allowing for more detailed observation of the patient's eye movements on the computer and enabling precise guidance on the patient's eye movements.

[0056]

[0057]

[0058]

[0059] 4.4) During the test, when the patient's eye movement speed suddenly becomes too fast or too slow, and the θ value is greater than 0.8 and less than 1, or greater than 0.2 and less than 0.3, three new plane layers (XY plane instantaneous N, XZ plane instantaneous N, ZY plane instantaneous N, where N is the number of instantaneous movements) will be opened to display the process of this instantaneous movement. The coordinates are transformed by an adaptive matrix transformation algorithm, and the plane scaling ratio is dynamically adjusted according to the number of currently displayed plane layers, and the coordinate position is displayed in real time. The person guiding the test can also manually adjust the scaling of each plane. After adjustment, the plane will recalculate the coordinate position and display ratio, which is convenient for detailed observation and comparison of the eye movement situation when the instantaneous eye movement is too fast.

[0060] 4.5) Furthermore, if the patient has performed this test before, multiple historical planes will be displayed during the test, showing the trajectory graphs of previously performed tests, the number of instantaneous movements, the average speed of excessively fast instantaneous movements, and the average speed of excessively slow instantaneous movements. The size of the historical planes is automatically adjusted based on the number of historical tests. The trajectory graph uses an adaptive matrix transformation algorithm based on the stored 3D coordinate system data collected by the VR device and performs 2D coordinate calculations according to the automatically adjusted plane size. Automatic adjustment means that when there are multiple planes, the size of the current multiple historical planes is automatically adjusted according to the current screen display size. For example, if the screen height is 1024 and there is one plane, then the height of this plane is 1024. If there are two planes, it is scaled to 1024 / 2 = 512 (height). The purpose of automatic adjustment is to facilitate longitudinal observation of patient data to intuitively reflect changes in the patient's eye movements.

[0061] It can compare historical and current test data based on historical and current 2D plane displays, providing data to understand eye movement test results and eye condition. Alternatively, it can automatically compare historical and current tests and provide relevant risk warnings, which may include potential risks to the patient, as shown in Table 1.

[0062] Table 1

[0063]

[0064] 5) Doctors can use the data displayed on the current test plane, the data displayed on the historical test plane, and the risk warning information as data to compare the current test with the historical test, and then combine the risk warning information to determine the eye movement problems that the patient has during the test, so as to diagnose and analyze the patient's condition.

[0065] In the technical solution of this application, eye-tracking coordinate data collected by the VR device can be synchronized to the computer. The computer then converts the eye-tracking coordinates in the VR device into eye-tracking plane coordinates or graphics that can be displayed on the computer. On the computer, a scaling ratio can be set for the display of eye-tracking coordinates for eye-tracking testing. During the testing process, an adaptive matrix transformation algorithm is used to dynamically adjust the scaling ratio on the computer, thereby achieving the effect of accurately displaying the testing process.

[0066] refer to Figure 3 As an implementation of the above method, this application provides an embodiment of an eye-tracking testing device, which is similar to... Figure 2 Corresponding to the method embodiment shown, in addition to the features described below, the device embodiment may also include [features related to...]. Figure 2 The method embodiments shown have the same or corresponding features or effects. This device can be specifically applied to various electronic devices.

[0067] like Figure 3 As shown, the eye-tracking testing device 300 of this embodiment includes: a sending unit 301, an acquisition unit 302, and a display unit 303. The sending unit 301 is configured to send prompting information to a head-mounted VR device, wherein the prompting information is used to prompt the target object about the test content during the eye-tracking test. The acquisition unit 302 is configured to acquire the 3D eye-tracking coordinates of the target object collected by the head-mounted VR device during the eye-tracking test. The display unit 303 is configured to convert the 3D eye-tracking coordinates into 2D coordinates to display the target object's eye-tracking status on a display interface at a set size.

[0068] Optionally, the display unit is also configured to: filter out invalid 3D coordinates from the eye-tracking 3D coordinates based on the eye movement speed to obtain valid 3D coordinates; map the valid 3D coordinates onto multiple two-dimensional planes for display, wherein each two-dimensional plane is used to display the eye movement from a visual angle, and any two two-dimensional planes display different visual angles.

[0069] Optionally, the display unit is also configured to: acquire the first eye-tracking 3D coordinates in the eye-tracking 3D coordinates. Second eye movement 3D coordinates Among them, the first eye movement 3D coordinates Second eye movement 3D coordinates These are the 3D coordinates of the eye movements of both eyes; based on the first eye movement 3D coordinates... Second eye movement 3D coordinates Determine the eye-tracking parameter θ used to represent the speed of eye movement: If the eye-tracking parameter θ is less than the first threshold or greater than the second threshold, the first eye-tracking 3D coordinates will be... Second eye movement 3D coordinates Invalid 3D coordinates are filtered out from eye-tracking 3D coordinates. The second threshold is greater than the first threshold; a larger eye-tracking parameter θ indicates faster eye movement. If the eye-tracking parameter θ is greater than or equal to the first threshold and less than or equal to the second threshold, the first eye-tracking 3D coordinates are... Second eye movement 3D coordinates Retained as valid 3D coordinates.

[0070] Optionally, the display unit is also configured to: when the eye movement parameter θ is greater than or equal to the first threshold and less than the third threshold, or greater than the fourth threshold and less than or equal to the second threshold, determine that the target object's eyes have moved instantaneously, and reopen multiple two-dimensional planes to display the eye movement situation, so as to reflect the comparison results before and after the instantaneous movement through the original multiple two-dimensional planes and the reopened multiple two-dimensional planes, wherein the third threshold is greater than the first threshold and less than the fourth threshold, and the fourth threshold is less than the second threshold.

[0071] Optionally, the display unit is also configured to: utilize the width w of the two-dimensional plane, the height h of the two-dimensional plane, the original size s of the eye in the two-dimensional plane, and the scaling factor m of the eye to determine the first eye-tracking 3D coordinates as effective 3D coordinates. Second eye movement 3D coordinates Perform mapping:

[0072] Set the first 2D coordinate (x) C y CThe second 2D coordinates (x, y) are mapped onto the first two-dimensional plane for display, and the second 2D coordinates (x, y) are mapped onto the first two-dimensional plane for display. C , z C The third 2D coordinate (z) is mapped onto the second two-dimensional plane for display, and the third 2D coordinate (z) is mapped onto the second two-dimensional plane for display. C y C It is then mapped onto a third two-dimensional plane for display.

[0073] Optionally, the display unit is also configured to: while displaying the effective 3D coordinates mapped to multiple two-dimensional planes, if the target object has previously undergone eye-tracking testing, display the trajectory of the historical eye-tracking test, the number of instantaneous movements, the speed of instantaneous excessively fast movements, and the speed of instantaneous excessively slow movements in the historical two-dimensional plane.

[0074] In this embodiment, the specific processing of the sending unit 301, the acquisition unit 302, and the display unit 303 of the eye-tracking testing device 300, and the resulting technical effects, can be referred to respectively. Figure 2 The relevant descriptions of steps 201, 202 and 203 in the corresponding embodiments will not be repeated here.

[0075] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0076] like Figure 4 The diagram shown is a block diagram of an electronic device for an eye-tracking testing method according to an embodiment of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0077] like Figure 4As shown, the electronic device includes one or more processors 401, a memory 402, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 4 Take a processor 401 as an example.

[0078] The memory 402 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause at least one processor to perform the eye-tracking testing method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the eye-tracking testing method provided in this application.

[0079] Memory 402, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the eye-tracking testing method in the embodiments of this application (e.g., attached...). Figure 3 The sending unit 301, the acquisition unit 302, and the display unit 303 are shown. The processor 401 executes various functional applications and data processing of the server by running non-transient software programs, instructions, and modules stored in the memory 402, thereby implementing the eye-tracking test method in the above method embodiments.

[0080] Memory 402 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the eye-tracking testing electronic device, etc. Furthermore, memory 402 may include high-speed random access memory, and may also include non-transient memory, such as at least one disk storage device, flash memory device, or other non-transient solid-state storage device. In some embodiments, memory 402 may optionally include memory remotely located relative to processor 401, and these remote memories may be connected to the eye-tracking testing electronic device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0081] The electronic device for the eye-tracking testing method may further include an input device 403 and an output device 404. The processor 401, memory 402, input device 403, and output device 404 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0082] Input device 403 can receive input digital or character information, and generate key signal inputs related to user settings and function control of the eye-tracking testing electronic device, such as a touchscreen, keypad, mouse, trackpad, touchpad, pointer, one or more mouse buttons, trackball, joystick, etc. Output device 404 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touchscreen.

[0083] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0084] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0085] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0086] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0087] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0089] The units described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a sending unit, an acquisition unit, and a display unit. The names of these units do not necessarily limit the specific unit itself.

[0090] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to: send prompting information to a head-mounted VR device, wherein the prompting information is used to prompt the patient about the test content during the eye-tracking test; acquire the 3D coordinates of the patient's eyeballs during the eye-tracking test collected by the head-mounted VR device; and convert the valid 3D coordinates in the eye-tracking 3D coordinates into 2D coordinates for display.

[0091] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. An eye-tracking testing system, the eye-tracking testing system comprising: A head-mounted VR device used to acquire the 3D coordinates of the target object's eye movements during eye-tracking tests; A computer device, which is communicatively connected to the head-mounted VR device, is used to convert the eye-tracking 3D coordinates into 2D coordinates so as to display the eye-tracking of the target object in the display interface of the computer device at a set size. When the target object is found to deviate from the test or the eye-tracking position is deviated in the display interface of the computer, the 2D coordinate dots in the three two-dimensional planes are displayed by zooming in or out. The computer device is also used for: Invalid 3D coordinates are filtered out from the eye-tracking 3D coordinates based on eye movement speed to obtain valid 3D coordinates; The effective 3D coordinates are mapped onto multiple two-dimensional planes for display, wherein each two-dimensional plane is used to display the eye movement from a visual perspective, and any two two-dimensional planes display different visual perspectives; The computer device is also used to obtain the first eye-tracking 3D coordinates in the eye-tracking 3D coordinates. Second eye movement 3D coordinates Among them, the first eye-tracking 3D coordinates and the second eye-tracking 3D coordinates These are the 3D coordinates of the eye movements for each eye; Based on the first eye-tracking 3D coordinates and the second eye-tracking 3D coordinates Determine eye-tracking parameters to represent eye movement speed : ； The eye movement parameters If the first eye-tracking 3D coordinates are less than the first threshold or greater than the second threshold, then... and the second eye-tracking 3D coordinates Invalid 3D coordinates are filtered out from the eye-tracking 3D coordinates, wherein the second threshold is greater than the first threshold, and the eye-tracking parameters The higher the value, the faster the eye movement speed; The eye movement parameters If the first eye-tracking 3D coordinates are greater than or equal to the first threshold and less than or equal to the second threshold, then... and the second eye-tracking 3D coordinates Retained as valid 3D coordinates; The computer device is also used for the eye movement parameters If the target object's eyes move instantaneously, and the first threshold is greater than or equal to the first threshold and less than the third threshold, or the fourth threshold is greater than or equal to the second threshold, then multiple two-dimensional planes are reopened to display the eye movement. The comparison results before and after the instantaneous movement are shown by comparing the original multiple two-dimensional planes with the reopened multiple two-dimensional planes. The third threshold is greater than the first threshold and less than the fourth threshold, and the fourth threshold is less than the second threshold.

2. An eye-tracking testing method, the method comprising: Sending prompts to a head-mounted VR device, wherein the prompts are used to prompt the target object with the test content during the eye-tracking test; Obtain the 3D coordinates of the target object's eye movement during the eye-tracking test collected by the head-mounted VR device; The eye-tracking 3D coordinates are converted into 2D coordinates so that the eye movement of the target object can be displayed in the display interface at a set size. When the target object is found to deviate from the test or the eye movement position is deviated in the computer display interface, the 2D coordinate dots in the three two-dimensional planes are displayed by zooming in or out. Converting the eye-tracking 3D coordinates to 2D coordinates, and then displaying the eye-tracking of the target object at a set size in the display interface, includes: Invalid 3D coordinates are filtered out from the eye-tracking 3D coordinates based on eye movement speed to obtain valid 3D coordinates; The effective 3D coordinates are mapped onto multiple two-dimensional planes for display, wherein each two-dimensional plane is used to display the eye movement from a visual perspective, and any two two-dimensional planes display different visual perspectives; Invalid 3D coordinates are filtered out from the eye-tracking 3D coordinates based on eye movement speed to obtain valid 3D coordinates, including: Obtain the first eye-tracking 3D coordinate in the eye-tracking 3D coordinates. Second eye movement 3D coordinates Among them, the first eye-tracking 3D coordinates and the second eye-tracking 3D coordinates These are the 3D coordinates of the eye movements for each eye; Based on the first eye-tracking 3D coordinates and the second eye-tracking 3D coordinates Determine eye-tracking parameters to represent eye movement speed : ； The eye movement parameters If the first eye-tracking 3D coordinates are less than the first threshold or greater than the second threshold, then... and the second eye-tracking 3D coordinates Invalid 3D coordinates are filtered out from the eye-tracking 3D coordinates, wherein the second threshold is greater than the first threshold, and the eye-tracking parameters The higher the value, the faster the eye movement speed; The eye movement parameters If the first eye-tracking 3D coordinates are greater than or equal to the first threshold and less than or equal to the second threshold, then... and the second eye-tracking 3D coordinates Retained as valid 3D coordinates; The effective 3D coordinates are mapped onto multiple 2D planes for display, including: The eye movement parameters If the target object's eyes move instantaneously, and the first threshold is greater than or equal to the first threshold and less than the third threshold, or the fourth threshold is greater than or equal to the second threshold, then multiple two-dimensional planes are reopened to display the eye movement. The comparison results before and after the instantaneous movement are shown by comparing the original multiple two-dimensional planes with the reopened multiple two-dimensional planes. The third threshold is greater than the first threshold and less than the fourth threshold, and the fourth threshold is less than the second threshold.

3. The method according to claim 2, wherein, The effective 3D coordinates are mapped onto multiple 2D planes for display, including: Using the width of the two-dimensional plane The height of the two-dimensional plane The original size s of the eye in the two-dimensional plane and the scaling factor of the eye. For the first eye-tracking 3D coordinates as valid 3D coordinates and the second eye-tracking 3D coordinates Perform mapping: ； First 2D coordinates Mapping to the first two-dimensional plane for display, and using the second 2D coordinates Mapping to a second two-dimensional plane for display, and using the third 2D coordinates It is then mapped onto a third two-dimensional plane for display.

4. The method according to any one of claims 2 to 3, wherein, While mapping the effective 3D coordinates onto multiple two-dimensional planes for display, the method also includes: If the target object has previously undergone eye-tracking testing, display the trajectory of the historical eye-tracking test, the number of instantaneous movements, the speed of excessively fast instantaneous movements, and the speed of excessively slow instantaneous movements in the historical two-dimensional plane.

5. An eye-tracking testing device, the device comprising: The sending unit is configured to send a prompt message to a head-mounted VR device, wherein the prompt message is used to prompt the target object with the test content during the eye-tracking test; The acquisition unit is configured to acquire the 3D coordinates of the eye movement of the target object during the eye movement test, collected by the head-mounted VR device; The display unit is configured to convert the eye-tracking 3D coordinates into 2D coordinates to display the eye-tracking of the target object in the display interface at a set size. When the target object is found to deviate from the test or the eye-tracking position is deviated in the computer display interface, the 2D coordinate dots in the three two-dimensional planes are displayed by zooming in or out. The display unit is used to filter out invalid 3D coordinates from the eye-movement 3D coordinates based on the eye movement speed to obtain valid 3D coordinates; and to map the valid 3D coordinates onto multiple two-dimensional planes for display, wherein each two-dimensional plane is used to display the eye movement from a visual angle, and any two two-dimensional planes display different visual angles. The display unit is used to obtain the first eye-tracking 3D coordinates in the eye-tracking 3D coordinate system. Second eye movement 3D coordinates Among them, the first eye movement 3D coordinates Second eye movement 3D coordinates These are the 3D coordinates of the eye movements of both eyes; based on the first eye movement 3D coordinates... Second eye movement 3D coordinates Determine eye-tracking parameters to represent eye movement speed : In eye movement parameters If the first eye movement coordinates are less than the first threshold or greater than the second threshold, then the first eye movement 3D coordinates will be... Second eye movement 3D coordinates Invalid 3D coordinates are filtered out from eye-tracking 3D coordinates, where the second threshold is greater than the first threshold, and the eye-tracking parameters... A higher value indicates a faster eye movement speed; in eye movement parameters If the first eye-tracking 3D coordinates are greater than or equal to the first threshold and less than or equal to the second threshold, then... Second eye movement 3D coordinates Retained as valid 3D coordinates; The display unit is used for displaying eye-tracking parameters. If the target object's eyes move instantaneously, the first threshold is greater than or equal to the first threshold and less than the third threshold, or the fourth threshold is greater than or equal to the second threshold. Multiple two-dimensional planes are then reopened to display the eye movement. The comparison between the original two-dimensional planes and the reopened two-dimensional planes is used to show the result before and after the instantaneous movement. The third threshold is greater than the first threshold and less than the fourth threshold, and the fourth threshold is less than the second threshold.

6. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 2-4.

7. A computer-readable storage medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 2-4.

Images