Device and method for eye movement tracking test of vr / ar equipment

Abstract

A kind of device and test method for VR / AR equipment eye movement tracking test, contain simulation eyeball module, pitch module, rotation module, pupil distance control module, drive module, drive control module, master control module;Master control module sets test task type and test track, rotation module and pitch module drive simulation eyeball module to realize rotation movement and pitch movement, attitude sensor and drive control module detect and control corresponding module to carry out error compensation to the pitch angle and rotation angle of simulation eyeball module, master control module compares current fixation point with the fixation point given by VR / AR equipment eye movement tracking system, realize the precision analysis of VR / AR equipment eye movement tracking system.The present application effectively simulates human eye movement, solves the problem that existing test device lacks actual motion image information, intuitively and accurately reflects the real movement of human eye, and improves the stability of VR / AR equipment eye movement tracking test.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual equipment, and more particularly, to a device and method for VR / AR equipment eye movement tracking test. BACKGROUND

[0002] With the rapid development of intelligent robots and display devices, the virtual reality (VR) and augmented reality (AR) industries are receiving more and more attention. Such near-eye display products belong to head-mounted devices, and the distance between the human eye and the display module is relatively close. Through optical technology, a virtual and enlarged image is realized in the user's near-eye range to provide intuitive image, video or text information to the user. Today, related applications have gradually been applied to the film and television, game, medical and other industries. Eye tracking devices are an important part of VR / AR and other near-eye display products, aiming to provide users with a more immersive interactive experience. The device converts eye movement into a data stream, which contains information such as pupil position, gaze vector and gaze point of each eye. Essentially, the eye tracking device is a decoder of eye movement, so a large number of eye tracking test tasks need to be performed on the device to improve the decoding speed and accuracy of eye movement.

[0003] The traditional VR / AR equipment eye movement tracking test method requires the subject to first fix the head with a headrest, then wear a VR / AR device with an eye movement tracking device, and then display a calibration point on a screen at a certain distance from the human eye to promote eye movement, so that the gaze point position coincides with the calibration point position, and the VR / AR equipment eye movement tracking device test is realized. However, the traditional eye movement tracking test method is time-consuming and causes visual fatigue to the subject, affecting the accuracy of the data obtained. At the same time, this test method requires a large number of different subjects to participate in the test to improve the robustness of the gaze tracking device. To solve this problem, a learning-based eye movement tracking test method is proposed, but this method requires a large number of data sets with pupil center coordinates, gaze vectors or gaze points as labels and static eye photos as features to train and verify the model, so it cannot intuitively and accurately reflect various actual movements of the human eye, such as gaze, saccade and pursuit.

[0004] Patent document CN117991502A proposes an eye movement tracking device and test device. The device connects a simulated eyeball to a first rotating part, and makes the pupil and eyeball center of the simulated eyeball form a certain angle with the rotating shaft of the first rotating part. At the same time, the second rotating part is connected to the first rotating part, and the pitch angle, horizontal rotation angle and included angle generated by the two rotating parts are used to simulate the movement of the human eye. However, this device does not consider the image information obtained by the human eye in actual movement, so there is still a large difference between it and the real human eye. SUMMARY

[0005] The present application aims to overcome the deficiencies of the prior art and provide a device and method for VR / AR equipment eye movement tracking test that can simulate the image information acquired by human eyes in actual movement and at the same time simulate various actual movements of human eyes such as fixation, saccade and pursuit. Through the pitching and rotating movement of the two simulated eyeball modules, the change of pupil distance, and image acquisition and processing, the human eyes with different interpupillary distances in various actual movements such as fixation, saccade and pursuit can be accurately simulated, and the images acquired by human eyes in these movements can be simulated at the same time. According to the simulated characteristics of human eyes, the acquired images during movement are processed and displayed, effectively solving the problem that the device for VR / AR equipment eye movement tracking test lacks image information acquired by human eyes in actual movement. Through the pitching transmission part and the rotating transmission part, the volume of the device is reduced, making the device easier to operate and use in the eye movement tracking test of small-volume VR / AR equipment, providing stronger portability for the eye movement tracking test of VR / AR equipment, and effectively solving the problem of large volume and low portability of the device for VR / AR equipment eye movement tracking test.

[0006] The technical solution of the present application is as follows:

[0007] In one aspect, the present application provides a device for VR / AR equipment eye movement tracking test, characterized in that it comprises:

[0008] a simulated eyeball module for simulating the movement of human eyes and configured with a posture sensor for real-time detection of the pitching angle and rotating angle of the simulated eyeball module;

[0009] a pitching module connected with the simulated eyeball module for driving the simulated eyeball module to realize pitching movement;

[0010] a rotating module connected with the simulated eyeball module for driving the simulated eyeball module to realize rotating movement;

[0011] an interpupillary distance control module for adjusting the simulated eyeball module to simulate the interpupillary distance of different users;

[0012] a driving module connected with the pitching module, rotating module and interpupillary distance control module respectively for providing power to perform corresponding movement control;

[0013] a driving control module connected with the posture sensor and driving module respectively for receiving the pitching angle and rotating angle information of the simulated eyeball module detected by the posture sensor and controlling the driving module to perform error compensation based on the information;

[0014] The master control module is connected with the driving control module, the attitude sensor and the VR / AR device eye tracking system respectively, is used for setting a test task type and a test track, receiving a gaze point given by the VR / AR device eye tracking system, comparing the gaze point currently acquired by the simulation eyeball module through the VR / AR device eye tracking system with the set gaze point, and realizing precision analysis on the VR / AR device eye tracking system.

[0015] Further, the simulation eyeball module comprises a first simulation eyeball module and a second simulation eyeball module, wherein the first simulation eyeball module and the second simulation eyeball module are composed of a simulation cornea, a simulation iris, a simulation sclera, a human eye lens, an image sensor and an attitude sensor, wherein the simulation cornea, the simulation iris, the human eye lens and the image sensor form a coaxial optical system with the human eye lens optical axis as the axis; the first simulation eyeball module and the second simulation eyeball module can reflect light generated by a light source of an eye tracking device in a VR / AR device to generate Purkinje spots for the eye tracking device to identify the spots to acquire eye movement data, receive instructions of the driving control module, adjust the pupil diameter and the focal length according to the test task type and the scene change respectively, simulate the human eye to adjust the light intensity into the eye according to the light intensity of the outside world, and facilitate the identification of the pupil by the eye tracking device in the VR / AR device; the image sensor clearly images an object located on the spatial gaze point coordinate at the current time and synchronously transmits the collected images during movement to the master control module for display, simulates the collection, processing and transmission of the light signals by the human eye retina; the attitude sensor synchronously measures the motion data of the first simulation eyeball module or the second simulation eyeball module and synchronously transmits the motion data to the master control module for angle compensation of the motion of the test device to accurately simulate the human eyes to watch the environment and different types of actual movements;

[0016] The pitch module comprises a first pitch module and a second pitch module; wherein the first pitch module comprises a first pitch driving component, a first rotation transmission component and a first pitch module support connecting the first pitch driving component and the first rotation transmission component; the second pitch module comprises a second pitch driving component, a second pitch transmission component and a second pitch module support connecting the second pitch driving component and the second rotation transmission component, the first pitch module is used for acquiring instructions of the driving module, driving the first simulation eyeball module to make a pitch movement around the pitch axis, simulating the actual movements of the human right eye fixation, saccade and pursuit; the second pitch module is used for acquiring instructions of the driving module, driving the second simulation eyeball module to make a pitch movement around the pitch axis, simulating the actual movements of the human left eye fixation, saccade and pursuit;

[0017] The rotation module comprises a first rotation module and a second rotation module, wherein the first rotation module is composed of a first rotation driving part and a first rotation transmission part, is used to obtain the instruction of the driving module, drive the first simulation eyeball module to make rotational movement around the rotation axis, and simulate the actual movement of the right eye of a human being; the second rotation module is composed of a second rotation driving part and a second rotation transmission part, is used to obtain the instruction of the driving module, drive the second simulation eyeball module to make rotational movement around the rotation axis, and simulate the actual movement of the left eye of a human being.

[0018] The interpupillary distance control module comprises a first rotation module base, a second rotation module base and an interpupillary distance controller, the first rotation module base is connected with the first rotation driving part, the second rotation module base is connected with the second rotation driving part, the interpupillary distance controller is connected with the first rotation module base, the second rotation module base and the driving module respectively, is used to adjust the pupil center distance of the first simulation eyeball module and the second simulation eyeball module according to the command or data of the driving module, simulate the interpupillary distance of different people, and the change range is 40mm to 80mm.

[0019] Further, the first pitching driving part, the first rotation driving part, the second pitching driving part and the second rotation driving part are direct current motors, stepping motors or servo motors; the first pitching transmission part, the first rotation transmission part, the second pitching transmission part and the second rotation transmission part are gear transmissions, worm and gear mechanisms, shafts, synchronous pulley sets, chain and sprocket sets or V-belt sets.

[0020] The image sensor is located in the back hemisphere of the simulation sclera and in the eye pupil box of the human eye lens; the simulation iris is a diaphragm with a non-white surface and an adjusting structure; the human eye lens has a focal length adjusting module and at least one positive power lens.

[0021] The simulation cornea is a convex-concave lens made of glass, transparent resin or silicon gel.

[0022] The simulation iris is a variable diaphragm with a mechanical adjusting structure, a variable diaphragm with an electric adjusting mechanism, a transmissive display, a transmissive LCD display screen, a transmissive LED display screen or a transmissive OLED display screen.

[0023] Furthermore, the driving module is used to acquire command programs or data from the driving control module and control corresponding modules or components to execute corresponding instructions; the driving control module is used to acquire command programs or data transmitted by the main control module, synchronously control the driving module to drive corresponding modules or components to execute instructions, and synchronously control the human eye lens and the simulated sclera in the first simulated eye module and the second simulated eye module to adjust focus and pupil size according to the test task and scene changes; at the same time, it synchronizes timing information with the main control module and synchronously sends the current spatial gaze point coordinate information to the main control module; the main control module is used to decompose the test trajectory into motion data of each module. The system synchronously sends commands to the drive control module to control each module. Simultaneously, it edits and processes the images input from the image sensors in the first and second simulated eye modules according to the test task to simulate the sensitivity of different human eyes to information such as color and brightness in the environment. Simultaneously, it acquires and displays the images transmitted by the image sensors in the first and second simulated eye modules, the current spatial gaze point coordinates, and the spatial gaze point coordinates given by the VR / AR device's eye-tracking device. Based on the motion data transmitted by the posture sensor, it synchronously controls the drive control module to perform angle compensation on the first and second simulated eye modules.

[0024] Secondly, the present invention provides a testing method for eye tracking in VR / AR devices, characterized by comprising the following steps:

[0025] S1. Place the VR / AR device under test in front of the testing device according to any one of claims 1 to 7, such that the simulated eye module is located inside the pupil box of the VR / AR device under test;

[0026] S2. Set the eye-tracking test task type and test trajectory for VR / AR devices through the main control module;

[0027] S3. Drive the pitch and rotation modules to make the simulated eye module pitch and rotate according to the set test trajectory;

[0028] S4. The pitch and rotation angles of the simulated eye module are detected in real time using an attitude sensor, and error compensation is performed through a drive control module;

[0029] S5. The main control module receives gaze point information from the eye-tracking system of the VR / AR device based on the simulated eye movement module.

[0030] S6. Compare the gaze point of the current simulated eye module with the gaze point given by the eye-tracking system of the VR / AR device, and analyze the accuracy of the eye-tracking system of the VR / AR device.

[0031] Furthermore, it also includes the following steps: adjusting the interpupillary distance of the simulated eye module through the interpupillary distance control module according to the testing requirements, so as to simulate the eye characteristics of different users for testing;

[0032] The image acquisition module is used to collect image information of the simulated eye module in the VR / AR environment in real time, so as to further analyze the performance of the eye tracking system of VR / AR device.

[0033] Furthermore, step S4 specifically includes:

[0034] The S4.1 attitude sensor acquires the angle data of the first and second simulated eye modules after the test task and test trajectory are set. The main control module compares the current angle data with the set initial angle data and controls the drive control module to reset the corresponding module.

[0035] The S4.2 main control module sends instructions to the drive control module according to the pupil center distance set in the test task, controls the pupil distance control module to adjust the pupil center distance of the two simulated eyeball modules, and according to the set VR / AR device eye tracking test trajectory, the trajectory is broken down into motion data of each module in time series.

[0036] The S4.3 main control module sends motion data to the drive control module in a sequential manner, so that the corresponding module drives the first simulated eye module and the second simulated eye module to perform pitch or rotation movements.

[0037] The S4.4 attitude sensor detects the pitch and rotation angles of the first and second simulated eye modules. The main control module compares the current rotation and pitch angles with the required rotation and pitch angles, so that the drive control module controls the corresponding modules to perform angle compensation.

[0038] S4.5 is repeated until the test trajectory is completed.

[0039] Compared with the prior art, the present invention has the following obvious and prominent substantive features and significant advantages:

[0040] 1. This invention utilizes a simulated eyeball module, a pitch module, a rotation module, and an interpupillary distance control module. Through the interconnection of these modules, it simulates the actual eye movements of humans under different interpupillary distances, such as fixation, saccades, and tracking. It synchronously acquires images collected during the simulated eyeball module's movement, as well as the three-dimensional coordinate information of the spatial gaze point and the three-dimensional coordinate information of the spatial gaze point provided by the VR / AR device. This effectively solves the problem that VR / AR device eye-tracking testing devices lack image information acquired from the actual movement of the human eye.

[0041] 2. This invention uses the pitch or rotation angles of two simulated eye modules obtained by the attitude sensor, and uses the main control module and drive control module to perform angle compensation, so as to achieve accurate simulation of the actual human eye movements such as gaze, saccade, and tracking. At the same time, by using the pitch transmission component in the pitch module and the rotation transmission component in the rotation module, the size of the device is reduced, effectively solving the problems of large size and low portability of VR / AR device eye tracking test devices.

[0042] 3. The testing method of this invention can perform eye-tracking tests on VR / AR devices according to different testing requirements. This method can initialize eye movement parameters such as starting angle, ending angle, movement speed, movement acceleration, uniform movement time, and accelerated movement time according to different test task types and test trajectories, and obtain the three-dimensional coordinates of the current spatial gaze point. These coordinates are then compared with the gaze point coordinates provided by the eye-tracking system in the VR / AR device to achieve accuracy analysis of the VR / AR device's eye-tracking system. Attached Figure Description

[0043] Appendix Figure 1 This is a schematic diagram of the structure of Embodiment 1 of the eye-tracking testing device and method for VR / AR devices of the present invention;

[0044] Appendix Figure 2 These are schematic diagrams of the simulated eye module in Embodiments 1 and 2 of the present invention for eye-tracking testing device and method for VR / AR devices;

[0045] Appendix Figure 3 This is a schematic diagram of the structure of Embodiment 2 of the eye-tracking testing device and method for VR / AR devices of the present invention;

[0046] Appendix Figure 4 A flowchart outlining an eye-tracking testing method for VR / AR devices provided in Embodiments 1 and 2 of the present invention;

[0047] Appendix Figure 5 The flowchart below provides an overview of the control method for an eye-tracking testing device for VR / AR equipment, as provided in Embodiments 1 and 2 of the present invention.

[0048] Appendix Figure 6 This is a schematic diagram providing an overview of an eye-tracking testing method for VR / AR devices, as provided in Embodiments 1 and 2 of the present invention.

[0049] In the diagram: 100 is the first simulated eyeball module, 101 is the simulated cornea, 102 is the simulated iris, 103 is the simulated sclera, 104 is the human eye lens, 105 is the image sensor, 110 is the first pitch module, 111 is the first pitch drive component, 112 is the first pitch transmission component, 113 is the first pitch module support, 120 is the first rotation module, 121 is the first rotation drive component, 122 is the first rotation transmission component, 130 is the interpupillary distance control module, and 131 is the first rotation module. Module base, 132 is the second rotating module base, 133 is the interpupillary distance controller, 140 is the drive module, 150 is the drive control module, 160 is the main control module, 200 is the second simulated eyeball module, 210 is the second pitch module, 211 is the second pitch drive component, 212 is the second pitch transmission component, 213 is the second pitch module bracket, 220 is the second rotation module, 221 is the second rotation drive component, 222 is the second rotation transmission component, and 300 is the VR / AR device.

[0050] It should be understood that the above figures are only schematic and are not drawn to scale. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] The above solution will be further described below with reference to specific embodiments. The preferred embodiments of the present invention are described in detail below: Embodiment 1

[0053] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of the eye-tracking testing device and method for VR / AR devices according to Embodiment 1 of the present invention. It includes a first simulated eye module 100, a first pitch module 110, a first pitch drive component 111, a first pitch transmission component 112, a first pitch module support 113, a first rotation module 120, a first rotation drive component 121, a first rotation transmission component 122, an interpupillary distance control module 130, a first rotation module base 131, a second rotation module base 132, a drive module 140, a drive control module 150, a main control module 160, a second simulated eye module 200, a second pitch module 210, a second pitch drive component 211, a second pitch transmission component 212, a second pitch module support 213, a second rotation module 220, a second rotation drive component 221, and a second rotation transmission component 222.

[0054] The first simulated eye module 100 and the second simulated eye module 200 are used to simulate the actual movements of a person's right and left eyes, respectively, and synchronously transmit the acquired image data and motion data to the main control module 160 to achieve accurate simulation of the human eye's viewing environment and different types of actual movements. Both the first simulated eye module 100 and the second simulated eye module 200 consist of a simulated cornea 101, a simulated iris 102, a simulated sclera 103, a human eyepiece lens 104, an image sensor 105, and a posture sensor 106.Both the first simulated eye module 100 and the second simulated eye module 200 have the following six functions: Firstly, the simulated cornea 101, simulated iris 102, human eye lens 104, and image sensor 105 in the first simulated eye module 100 and the second simulated eye module 200 form a coaxial optical system, with the optical axis of the system being the optical axis of the human eye lens 104, to simulate the imaging structure in the human eye; secondly, the simulated cornea 101 in the first simulated eye module 100 and the second simulated eye module 200 can reflect the light generated by the light source of the eye-tracking device in the VR / AR device, producing a light output. The light spot facilitates the eye-tracking device in the VR / AR device to identify the light spot and obtain eye movement data; furthermore, the simulated iris 102 in the first simulated eye module 100 and the second simulated eye module 200 can synchronously receive instructions from the drive control module 150, and adjust the pupil diameter of the first simulated eye module 100 or the second simulated eye module 200 according to the test task type and scene changes, simulating the human eye adjusting the intensity of light entering the eye according to the intensity of external light, and facilitating the eye-tracking device in the VR / AR device to identify the pupil and obtain the pupil center coordinates and pupil diameter; furthermore, the first simulated The human eye lens 104 in the eye module 100 and the second simulated eye module 200 can synchronously receive instructions from the drive control module 150, and adjust the focal length of the first simulated eye module 100 or the second simulated eye module 200 according to the test task type and scene changes, so that the image sensor 105 in the first simulated eye module 100 and the second simulated eye module 200 can clearly image the object located at the spatial gaze point coordinates at the current time; furthermore, the image sensor 105 in the first simulated eye module 100 and the second simulated eye module 200 can clearly image the object located at the spatial gaze point coordinates at the current time; and the image sensor 105 in the first simulated eye module 100 and the second simulated eye module 200 can also clearly image the object located at the spatial gaze point coordinates at the current time. Images acquired during the movement of the first simulated eye module 100 and the second simulated eye module 200 are synchronously transmitted to the main control module 160 for display, simulating the acquisition, processing, and transmission of light signals by the human retina; fifth, the attitude sensor 106 in the first simulated eye module 100 and the second simulated eye module 200 can synchronously measure the motion data of the first simulated eye module 100 or the second simulated eye module 200 before the start of the test task and during the test, and synchronously transmit the motion data to the main control module 160, and perform angle compensation for the motion synchronization of the VR / AR eye tracking test device according to the test task.Preferably, the maximum pitch angle and maximum rotation angle of the first simulated eye module 100 and the second simulated eye module 200 are ±45 degrees. When the first simulated eye module 100 and the second simulated eye module 200 perform pitch or rotation movements, the maximum angular velocity is 120 rpm and the maximum angular acceleration is 20π rad / s². More preferably, the pitch angle range of the first simulated eye module 100 and the second simulated eye module 200 is ±20 degrees to ±45 degrees, and the rotation angle range is ±20 degrees to ±75 degrees, so that the testing device can realistically simulate the eye movement angles of a normal human eye.

[0055] The first pitch module 110 is used to receive instructions from the drive module 140, causing the first simulated eye module 100 to perform pitch motion around the pitch axis, thereby simulating actual human right eye movements such as fixation, saccades, and tracking. The first pitch module 110 consists of a first pitch drive component 111 and a first pitch transmission component 112. Preferably, the first pitch module 110 also has an error compensation device, such as a photoelectric encoder, Hall sensor, magnetoresistive sensor, or inertial sensor, for closed-loop control of the VR / AR eye-tracking testing device. More preferably, the first pitch module 110 has a matte black surface coating, facilitating the VR / AR device's eye-tracking device to recognize the first simulated eye module 100 for eye-tracking testing.

[0056] The first pitch drive component 111 can be a DC motor, a stepper motor, or a servo motor. The first pitch drive component 111 is connected to the first pitch transmission component 112, and the connection method can be a key connection or a coupling connection.

[0057] The first pitch transmission component 112 can be a gear transmission, a worm gear mechanism, or a shaft. The first pitch transmission component 112 is connected to the simulated sclera 103 in the first simulated eyeball module 100, and the connection method can be a coupling connection or a key connection.

[0058] The first pitch module bracket 113 connects the first pitch drive component 111 and the first rotation transmission component 122, enabling the first simulated eye module 100 to rotate around a rotation axis, thus simulating actual human right eye movements such as fixation, saccades, and tracking. The first pitch module bracket 113 can be made of metal, engineering plastic, or a 3D printed material. The first pitch module bracket 113 is connected to the first pitch drive component 111 via threaded connection, welding, or as an integral part of the first pitch drive component 111. The first pitch module bracket 113 is also connected to the first rotation transmission component 122 via flange connection, welding, or as an integral part of the first rotation transmission component 122. Preferably, the first pitch module bracket 113 is an L-shaped plate with a matte black surface coating, facilitating the eye-tracking device of the VR / AR device to identify the first simulated eye module 100 for eye-tracking testing.

[0059] The first rotation module 120 is used to receive instructions from the drive module 140 and provide rotational motion to the first simulated eye module 100 to simulate actual human right eye movements such as fixation, saccades, and tracking. The first rotation module 120 consists of a first rotation drive component 121 and a first rotation transmission component 122. Preferably, the first rotation module 120 also has an error compensation device, such as a photoelectric encoder, Hall sensor, magnetoresistive sensor, or inertial sensor, for closed-loop control of the VR / AR eye-tracking testing device. More preferably, the first rotation module 120 has a matte black surface coating, facilitating the VR / AR device's eye-tracking device to recognize the first simulated eye module 100 for eye-tracking testing.

[0060] The first rotary drive component 121 can be a DC motor, a stepper motor, or a servo motor. The first rotary drive component 121 is connected to the first rotary transmission component 122, and the connection method can be a key connection or a coupling connection. The first rotary drive component 121 is also connected to the first rotary module base 131, and the connection method can be a threaded connection, welding, or integral with the first rotary module base 131.

[0061] The first rotary transmission component 122 can be a gear transmission, a worm gear mechanism, or a shaft. The first rotary transmission component 122 is connected to the first pitch module bracket 113. The connection method can be a flange connection, welding, or integral with the first rotary transmission component 122, so that the first simulated eyeball module 100 rotates around its rotation axis.

[0062] The interpupillary distance control module 130 receives commands or data from the drive module 140 and adjusts the pupil center distance between the first simulated eye module 100 and the second simulated eye module 200 according to the commands or data to simulate the interpupillary distance of different people. The interpupillary distance control module 130 consists of a first rotating module base 131, a second rotating module base 132, and an interpupillary distance controller 133. The interpupillary distance control module 130 controls the range of change of the pupil center distance between the first simulated eye module 100 and the second simulated eye module 200 to be from 40mm to 80mm. Preferably, the interpupillary distance control module 130 controls the range of change of the interpupillary distance to be from 56mm to 64mm to match the interpupillary distance of a normal human eye. More preferably, the interpupillary distance control module 130 also has an error compensation device, such as a photoelectric encoder, a Hall sensor, a magnetoresistive sensor, or an inertial sensor, and is connected to the main control module 160 for closed-loop control of the VR / AR eye-tracking testing device. More preferably, the interpupillary distance control module 130 also has a matte black surface coating, which facilitates the VR / AR device's eye-tracking device to identify the first simulated eye module 100 and the second simulated eye module 200, so as to facilitate eye-tracking testing.

[0063] The first rotating module base 131, which can be a platform made of metal or engineering plastic, is used to fix the first rotating drive component 121 onto the interpupillary distance control module 130. The first rotating module base 131 is connected to the first rotating drive component 121, and the connection method can be threaded connection, welding, or integral with the first rotating drive component 121.

[0064] The second rotating module base 132, which can be a platform made of metal or engineering plastic, is used to fix the second rotating drive component 221 onto the interpupillary distance control module 130. The second rotating module base 132 is connected to the second rotating drive component 221, and the connection method can be threaded connection, welding, or integral with the second rotating drive component 221.

[0065] The interpupillary distance controller 133 can be a linear motor, a ball screw, or a displacement stage. The interpupillary distance controller 133 is connected to the first rotary module base 131 via a key connection, coupling connection, or threaded connection. The interpupillary distance controller 133 is also connected to the second rotary module base 132 via a key connection, coupling connection, or threaded connection. The interpupillary distance controller 132 is connected to the drive control module 150 via USB, serial port, or general-purpose I / O.

[0066] The drive module 140 is used to acquire command programs or data from the drive control module 150 and control corresponding modules or components to execute corresponding instructions. The drive module 140 is the host computer for the first pitch drive component 111, the first rotation drive component 121, the interpupillary distance controller 133, the second pitch drive component 211, and the second rotation drive component 221, and the slave computer for the drive control module 150. The drive module 140 can be a stepper motor drive board, a servo motor drive board, an ARM chip, a DSP chip, an FPGA chip, a microcontroller, an industrial control computer, etc. The drive module 140 is connected to the first pitch drive component 111, the first rotation drive component 121, the interpupillary distance controller 133, the second pitch drive component 211, the second rotation drive component 221, and the drive control module 150, respectively, and the connection method can be wire soldering, crimping, plug-in connection, USB, serial port, parallel port, etc.

[0067] The drive control module 150 serves as both the host computer of the drive module 140 and the slave computer of the main control module 160, and includes a component for generating synchronization signals. The drive control module has three main functions: firstly, it acquires command programs or data transmitted by the main control module 160 and synchronously controls the drive module 140 to drive corresponding modules or components to execute instructions; secondly, based on test tasks and scene changes, it synchronously controls the human eye lens 104 and the simulated sclera 102 in the first simulated eye module 100 and the second simulated eye module 200 to adjust focus and pupil size; and thirdly, it synchronizes timing information with the main control module 160, synchronously sending the current spatial gaze point coordinates to the main control module 160. The drive control module 150 can be a server, an ARM chip, a DSP chip, an FPGA chip, a microcontroller, or an industrial control computer, etc. The component generating the synchronization signal in the drive control module 150 can be a crystal oscillator, a clock chip, or a pulse generator. The drive control module 150 is connected to the simulated sclera 102 and the human eye lens 104 in the first simulated eyeball module 100 and the second simulated eyeball module 200, respectively. The connection method can be wired, such as via USB, serial port, or parallel port, or wireless, such as via Bluetooth, LAN, or LoRa. The drive control module 150 is connected to the drive module 140, and the connection method can be wire soldering, crimping, or plug-in connection. The drive control module 150 is connected to the main control module 160, and the connection method can be wired, such as via USB, serial port, or parallel port, or wireless, such as via Bluetooth, LAN, or LoRa. The timing synchronization between the drive control module 150 and the main control module 160 can be hardware-level synchronization, such as using the synchronization signal generated by the synchronization signal generation component in the drive control module 140, or software-level synchronization, such as using a clock synchronization program or network time synchronization.

[0068] The main control module 160 is the host computer of the drive control module 150. The main control module has three functions: First, based on the test task and test trajectory, the main control module 160 decomposes the test trajectory into motion data for each module and synchronously sends commands to the drive control module 150 to control each module. Simultaneously, it edits the images input from the image sensors 105 in the first simulated eye module 100 and the second simulated eye module 200 according to the test task, such as changing parameters like brightness, contrast, saturation, gamma value, and resolution of the input images to simulate the sensitivity of different human eyes to color, brightness, and other information in the environment. Second, the main control module 160... The main control module 160 synchronously acquires and displays the images transmitted by the image sensors 105 in the first simulated eye module 100 and the second simulated eye module 200, the current spatial gaze point coordinates, and the spatial gaze point coordinates given by the eye-tracking device of the VR / AR device. On the other hand, the main control module 160 synchronously controls the drive control module 150 to perform angle compensation on the first simulated eye module 100 and the second simulated eye module 200 based on the motion data transmitted by the attitude sensors 106 in the first simulated eye module 100 and the second simulated eye module 200. The main control module 160 can be a personal computer, a server, an ARM chip, a DSP chip, an FPGA chip, a microcontroller, or an industrial control computer, etc. The main control module 160 is connected to the image sensors 105 in the first simulated eye module 100 and the second simulated eye module 200, and the connection method can be through video interfaces such as VGA, HDMI, DVI, and DisplayPort, USB, serial ports, and general I / O. The main control module 160 is connected to the attitude sensor 106 in the first simulated eye module 100 and the second simulated eye module 200. The connection method can be a wired connection such as USB, serial port, parallel port, etc., or a wireless connection such as Bluetooth, local area network, LoRa, etc. The main control module 160 is connected to the drive control module 150. The connection method can be a wired connection such as USB, serial port, parallel port, etc., or a wireless connection such as Bluetooth, local area network, LoRa, etc.

[0069] The second pitch module 210 is used to acquire instructions from the drive module 140 and provide pitch motion to the second simulated eye module 200, thereby simulating actual movements of the human left eye, such as fixation, saccades, and tracking. The second pitch module 210 consists of a second pitch drive component 211 and a second pitch transmission component 212. Preferably, the second pitch module 210 also has an error compensation device, such as a photoelectric encoder, Hall sensor, magnetoresistive sensor, or inertial sensor, for closed-loop control of the VR / AR eye-tracking testing device. More preferably, the second pitch module 210 has a matte black surface coating, facilitating the VR / AR device's eye-tracking device to recognize the two simulated eye modules 200 for eye-tracking testing.

[0070] The second pitch drive component 211 can be a DC motor, a stepper motor, or a servo motor. The second pitch drive component 211 is connected to the second pitch transmission component 212, and the connection method can be a key connection or a coupling connection. The second pitch drive component 211 is also connected to the second pitch module bracket 213, and the connection method can be a threaded connection, welding, or integral with the second pitch module bracket 213.

[0071] The second pitch transmission component 212 can be a gear transmission, a worm gear mechanism, or a shaft. The second pitch transmission component 212 is connected to the simulated sclera 103 in the second simulated eyeball module 200, and the connection method can be a coupling connection or a key connection.

[0072] The second pitch module bracket 213 connects the second pitch drive component 211 and the second rotation transmission component 222, enabling the second simulated eye module 200 to rotate around a rotation axis, simulating various actual human eye movements such as fixation, saccades, and tracking. The second pitch module bracket 213 can be made of metal, engineering plastic, or 3D printed material. The second pitch module bracket 213 is connected to the second pitch drive component 211 via threaded connection, welding, or as an integral part of the second pitch drive component 211. The second pitch module bracket 213 is also connected to the second rotation transmission component 222 via flange connection, welding, or as an integral part of the second rotation transmission component 222. Preferably, the second pitch module bracket 213 is an L-shaped plate with a matte black surface coating, facilitating the eye-tracking device of the VR / AR device to identify the second simulated eye module 200 for eye-tracking testing.

[0073] The second rotation module 220 is used to receive instructions from the drive module 140 and provide rotational motion to the second simulated eye module 200, thereby simulating actual movements of the human left eye, such as fixation, saccades, and tracking. The second rotation module 220 consists of a second rotation drive component 221 and a second rotation transmission component 222. Preferably, the second rotation module 220 also has an error compensation device, such as a photoelectric encoder, Hall sensor, magnetoresistive sensor, or inertial sensor, for closed-loop control of the VR / AR eye-tracking testing device. More preferably, the second rotation module 220 has a matte black surface coating, facilitating the VR / AR device's eye-tracking device to recognize the second simulated eye module 200 for eye-tracking testing.

[0074] The second rotary drive component 221 can be a DC motor, a stepper motor, or a servo motor. The second rotary drive component 221 is connected to the second rotary transmission component 222, and the connection method can be a key connection or a coupling connection. The second rotary drive component 221 is also connected to the second rotary module base 132, and the connection method can be a threaded connection, welding, or integral with the second rotary module base 132.

[0075] The second rotary transmission component 222 can be a gear transmission, a worm gear mechanism, or a shaft. The second rotary transmission component 222 is connected to the second pitch module bracket 213. The connection method can be a flange connection, welding, or integral with the second rotary transmission component 222, so that the second simulated eyeball module 200 rotates around its rotation axis.

[0076] Figure 2 The diagram shows the structure of the simulated eye module in Embodiments 1 and 2 of the present invention for eye tracking testing device and method for VR / AR devices. It includes a simulated cornea 101, a simulated iris 102, a simulated sclera 103, a human eye lens 104, an image sensor 105, and a posture sensor 106.

[0077] A simulated cornea 101 is used to simulate the refractive function of the human cornea and to transmit light into the simulated sclera 103. Simultaneously, it reflects light generated by the light source of the eye-tracking device in the VR / AR device, producing a Pulcin spot, which is easily identified by the eye-tracking device in the VR / AR device to obtain eye-tracking data. The simulated cornea 101 can be made of glass, transparent resin, or a convex / concave lens made of silicone gel. The simulated cornea 101 is located outside the anterior hemisphere of the simulated sclera 103 and is connected to the simulated sclera 103. The connection method can be adhesive bonding, snap-fit, or integral with the simulated sclera 103. Preferably, the simulated cornea 101 is made of a material with a refractive index greater than or equal to 1.1 and a light transmittance greater than 70%. The anterior surface curvature radius of the simulated cornea 101 is less than 9 mm, the posterior surface curvature radius is less than or equal to 8 mm, the anterior surface refractive power is less than or equal to +50 D, the posterior surface refractive power is greater than or equal to -7 D, and the transverse and longitudinal diameters of the simulated cornea 101 are less than or equal to 13 mm. More preferably, the material used to fabricate the simulated cornea 101 has a refractive index ranging from 1.37 to 1.38, a light transmittance ranging from 97% to 98%, an anterior surface radius of curvature ranging from 7.7 mm to 7.9 mm, a posterior surface radius of curvature ranging from 6.7 mm to 6.9 mm, an anterior surface refractive power ranging from +48D to +49D, a posterior surface refractive power ranging from -5D to -6D, a total refractive power ranging from +42D to +44D, a transverse diameter ranging from 11 mm to 13 mm, and a longitudinal diameter ranging from 10.5 mm to 11.5 mm, to match the corneal shape and refractive power of a normal human eye, thereby more accurately simulating the human cornea.

[0078] A simulated iris 102 is used to simulate the color of the human eye's iris and the changes in pupil diameter. The simulated iris 102 has two functions: firstly, it receives commands from the drive control module 150 in a sequential manner, adjusting the pupil diameter of the first simulated eye module 100 or the second simulated eye module 200 according to the test task type and scene changes, simulating the human eye adjusting the intensity of incoming light according to the intensity of external light; secondly, it is used by the eye-tracking device in VR / AR devices to identify the pupil, facilitating the acquisition of the pupil center coordinates and pupil diameter by the eye-tracking device. The simulated iris 102 is a non-white aperture with an adjustable structure. It can be a variable aperture with a mechanical adjustment structure, a variable aperture with an electrically adjustable mechanism, or a transmissive display, where the transmissive display can be a transmissive LCD screen, a transmissive LED screen, or a transmissive OLED screen. The simulated iris 102 is located between the simulated cornea 101 and the human eye lens 104, and is connected to the human eye lens 104. The connection method can be using thin transparent adhesive or a cage plate / rod connection. The simulated iris 102 is connected to the drive control module 150, and the connection method can be a wired connection such as USB, serial port, parallel port, etc., or a wireless connection such as Bluetooth, local area network, LoRa, etc. Preferably, the simulated pupil variation range simulated by the simulated iris 102 is 2mm to 7mm, thereby more accurately simulating the function of the human eye iris, which facilitates the eye-tracking device in the VR / AR device to perform edge detection and recognition of the pupil, iris, and sclera of the first simulated eye module 100 or the second simulated eye module 200 to obtain information such as pupil center coordinates, eyeball optical axis vector, and visual axis vector.

[0079] A simulated sclera 103 is used to simulate the protection of the human eyeball by the sclera. It houses the simulated iris 102, the human eye lens 104, and the image sensor 105, protecting these components from external environmental influences. Together with the simulated iris 102, it is used in VR / AR devices for eye-tracking devices to identify the pupil edge and obtain the pupil center coordinates and pupil diameter. The simulated sclera 103 can be a white object made of metallic or non-metallic materials. A spatial coordinate system is established using the rotation center of the first simulated eyeball module 100 or the second simulated eyeball module 200 as the origin O, and the pitch axis of the first simulated eyeball module 100 or the second simulated eyeball module 200 as the x-axis, following a right-hand screw. Using the z-axis as a reference, the simulated sclera 103 can be divided into an anterior hemisphere in the +z direction and a posterior hemisphere in the -z direction. The simulated sclera 103 is connected to the simulated cornea 101, and the connection method can be glued, snap-fitted, or integrated with the simulated sclera 103. The simulated sclera 103 is connected to the first pitch transmission component 112 or the second pitch transmission component 212, and the connection method can be a coupling connection or a key connection. The simulated sclera 103 is connected to the image sensor 105, and the connection method can be glue connection, snap-fit ​​connection, or integral connection with the simulated sclera 103. The simulated sclera 103 is connected to the attitude sensor 106, and the connection method can be snap-fit ​​connection or glue connection. Preferably, the simulated sclera 103 is an opaque white spherical shell with an outer surface diameter ranging from 23mm to 26mm and a base radius ranging from 10mm to 13mm, thereby more accurately matching the surface transverse diameter and surface longitudinal diameter of the simulated cornea 101, facilitating the eye-tracking device in the VR / AR device to perform edge detection and recognition of the pupil, iris, and sclera of the first simulated eye module 100 or the second simulated eye module 200 to obtain information such as pupil center coordinates, eyeball optical axis vector, and visual axis vector. More preferably, the simulated sclera 103 can also be a drum-shaped body with an outer surface diameter ranging from 23mm to 26mm and a shell thickness ranging from 2mm to 3mm, with the radius of the anterior hemisphere crown ranging from 10mm to 13mm and the radius of the posterior hemisphere crown ranging from 11mm to 12mm, so as to facilitate the fixation of the image sensor 105 to the simulated sclera 103 and the connection of the image sensor 105 to the main control module 160.

[0080] The human eyepiece lens 104, together with the simulated cornea 101, the simulated iris 102, and the image sensor 105, forms a coaxial optical system, with the optical axis of the system being the optical axis of the human eyepiece lens 104. The human eyepiece lens 104 has two functions: firstly, it is located in front of the image sensor 105 to simulate the process of the lens projecting onto the retina in the human eye; secondly, it has a module capable of adjusting the focal length. By synchronously receiving instructions from the drive control module 150, it can adjust the focal length of the first simulated eye module 100 or the second simulated eye module 200 according to the test task type and scene changes, enabling the image sensor 105 to clearly image objects located at the current spatial gaze point coordinates. The human eye lens 104 can be made of glass, transparent plastic, or fluorite. The lens assembly can be a lens-mirror combination structure consisting of multiple single lenses or mirrors, such as an in-line front diaphragm eyepiece lens group, a supercenter in-line front diaphragm lens group, or a zigzag front diaphragm lens-mirror group, or it can be a single lens structure. The human eye lens 104 includes at least one positive power lens. The module for focal length adjustment in the human eye lens 104 can be a liquid crystal lens, an electrowetting liquid lens, a liquid-filled liquid lens, or a lens assembly with a mechanical zoom structure. The focal length adjustment of the human eye lens 104 can be manual or electric. The human eye lens 104 is connected to the simulated iris 102, either using thin transparent adhesive or a cage plate / rod connection. The human eye lens 104 is connected to the image sensor 105, either using a cage plate / rod connection, a snap-fit ​​connection, or as an integral part of the image sensor 105. The human eye lens 104 is connected to the drive control module 150. The connection method can be wired, such as USB, serial port, parallel port, etc., or wireless, such as Bluetooth, LAN, LoRa, etc. Preferably, the field of view θ of the human eye lens 104 is ±30° to ±90°, the MTF cutoff frequency is greater than 40lp / mm, the Strehl ratio (SR) is greater than 0.15, the entrance pupil diameter is 2mm to 7mm, the field of view is ±60° to ±80°, the exit pupil distance is 10mm to 15mm, the front focal length is 10mm to 15mm, and the back focal length is 20mm to 30mm, so as to more realistically simulate the optimal viewing angle of the human eye.

[0081] An image sensor 105 is used to receive light signals from the human eyepiece lens 104 and convert them into images. It synchronously transmits the images acquired by the first simulated eyeball module 100 and the second simulated eyeball module 200 during movement to the main control module 160 for display, simulating the acquisition, processing, and transmission of light signals by the human retina. The image sensor 105 is located within the posterior hemisphere of the simulated sclera 103 and within the pupil box of the human eyepiece lens. The image sensor 105 can be planar or curved. The pixels of the image sensor 105 can be uniformly distributed or non-uniformly distributed. The image sensor 105 is connected to the human eyepiece lens 104, either by using a cage plate / rod connection, a snap-fit ​​connection, or as an integral part of the human eyepiece lens 104. The image sensor 105 is also connected to the simulated sclera 103, either by adhesive bonding, a snap-fit ​​connection, or as an integral part of the simulated sclera 103. The image sensor 105 is connected to the main control module 160, and the connection method can be through video interfaces such as VGA, HDMI, DVI, and DisplayPort, USB, serial port, and general I / O. Preferably, the frame rate of the image sensor 105 can vary between 24 frames per second and 240 frames per second, and its pixel density can decrease from the center to the periphery, with the central viewing area pixel density ranging from 300ppi to 450ppi, the peripheral viewing area ranging from 250ppi to 300ppi, and the radius of the central viewing area ranging from 1.4mm to 1.6mm, to simulate the equivalent pixel density and light-sensing effect of the fovea region and other areas of the human eye's retina. More preferably, the maximum circumscribed circle radius of the image sensor 105 should be less than 13mm to simulate the horizontal and vertical diameters of the human eye's retina within the eyeball. More preferably, the shutter speed of the image sensor 105 should be less than or equal to 1 / 43200s to match the limiting angle of resolution of the human eye.

[0082] The attitude sensor 106 can be an inertial sensor, an angle sensor, or a gyroscope. The attitude sensor 106 has three functions: First, it synchronously measures the motion data of the first simulated eye module 100 or the second simulated eye module 200, such as rotation angle, pitch angle, rotational angular velocity, pitch angular velocity, rotational angular acceleration, and pitch angular acceleration. Second, it synchronously transmits the motion data to the main control module 160 for angle compensation of the VR / AR eye-tracking testing device according to the test task. Third, after the test task is determined, the attitude sensor 106 can acquire the angle data of the first simulated eye module 100 and the second simulated eye module 200 after the completion of the previous test task and transmit this angle data to the main control module 160 for device reset. The attitude sensor 106 is located within the posterior hemisphere of the simulated sclera and behind the image sensor. The attitude sensor 106 is connected to the simulated sclera 103, and the connection method can be either a snap-fit ​​or an adhesive connection. The attitude sensor 106 is connected to the main control module 160. The connection method can be wired, such as USB, serial port, parallel port, etc., or wireless, such as Bluetooth, LAN, LoRa, etc. Preferably, the attitude sensor 106 can be an inertial measurement unit with a maximum zero-point offset range of less than or equal to ±0.1%, a zero-bias stability range of less than or equal to 0.01° / h, a sensitivity range of less than or equal to 10mV / ° / s, and a scaling factor error of less than or equal to 0.5%, so as to accurately control the rotation angle and pitch angle of the first simulated eye module 100 or the second simulated eye module 200.

[0083] In this embodiment, the VR / AR device eye-tracking testing device can perform vertical pitch movement, horizontal rotation movement, pupil distance change, and image acquisition and processing on two simulated eyeball modules. It can accurately simulate various actual human eye movements such as fixation, saccades, and tracking at different pupil distances, and process and display the images acquired during the movement according to the simulated human eye characteristics.

[0084] Example 2

[0085] An embodiment of the present invention for an eye-tracking testing device and method for VR / AR devices, such as... Figure 3As shown, the eye-tracking testing device for VR / AR devices includes a first simulated eye module 100, a first pitch module 110, a first pitch drive component 111, a first pitch transmission component 112, a first pitch module bracket 113, a first rotation module 120, a first rotation drive component 121, a first rotation transmission component 122, an interpupillary distance control module 130, a first rotation module base 131, a second rotation module base 132, a drive module 140, a drive control module 150, a main control module 160, a second simulated eye module 200, a second pitch module 210, a second pitch drive component 211, a second pitch transmission component 212, a second pitch module bracket 213, a second rotation module 220, a second rotation drive component 221, and a second rotation transmission component 222.

[0086] The first simulated eye module 100 and the second simulated eye module 200 are used to simulate the actual movements of a person's right and left eyes, respectively, and synchronously transmit the acquired image data and motion data to the main control module 160. Both the first simulated eye module 100 and the second simulated eye module 200 consist of a simulated cornea 101, a simulated iris 102, a simulated sclera 103, a human eyepiece lens 104, an image sensor 105, and a posture sensor 106.

[0087] The first pitch module 110 is used to acquire instructions from the drive module 140 and provide pitch motion for the first simulated eye module 100. The first pitch module 110 consists of a first pitch drive component 111 and a first pitch transmission component 112.

[0088] The first pitch drive component 111 is connected to the first pitch module bracket 113. The first pitch drive component 111 is connected to the first pitch transmission component 112, and the connection method can be a key connection or a coupling connection.

[0089] The first pitch transmission component 112 can be a synchronous belt pulley assembly, a chain and sprocket assembly, or a V-belt pulley assembly. The first pitch transmission component 112 is connected to the simulated sclera 103 in the first simulated eyeball module 100, and the connection method can be a coupling connection or a key connection. Preferably, the distance between the central axis of the input wheel and the central axis of the output wheel of the first pitch transmission component 112 is 16cm to 30cm.

[0090] The first pitch module bracket 113 is connected to the first pitch drive component 111 and the first rotation transmission component 122, so that the first simulated eyeball module 100 rotates around the rotation axis and the pitch axis.

[0091] The first rotation module 120 is used to receive instructions from the drive module 140 and provide rotational motion to the first simulated eyeball module 100. The first rotation module 120 consists of a first rotation drive component 121 and a first rotation transmission component 122.

[0092] The first rotary drive component 121 is connected to the first rotary module base 131. The first rotary drive component 121 is connected to the first rotary transmission component 122, and the connection method can be a key connection or a coupling connection.

[0093] The first rotary transmission component 122 can be a synchronous belt pulley assembly, a chain and sprocket assembly, or a V-belt pulley assembly. The first rotary transmission component 122 is connected to the first pitch module bracket 113, and the connection method can be a flange connection, welding, or integral with the first rotary transmission component 122, allowing the first simulated eyeball module 100 to rotate around its rotation axis. Preferably, the distance between the central axis of the input wheel and the central axis of the output wheel of the first rotary transmission component 122 is 16cm to 30cm.

[0094] The pupillary distance control module 130 provides different pupil center distances for the first simulated eye module 100 and the second simulated eye module 200. The pupillary distance control module 130 consists of a first rotating module base 131, a second rotating module base 132, and a pupillary distance controller 133. The first rotating module base 131 is used to fix the first rotating drive component 121 onto the pupillary distance control module 130. The first rotating module base 131 is connected to the first rotating drive component 121. The second rotating module base 132 is used to fix the second rotating drive component 221 onto the pupillary distance control module 130. The second rotating module base 132 is connected to the second rotating drive component 221. The pupillary distance controller 133 receives commands from the drive control module 150 and adjusts the pupil center distances of the first simulated eye module 100 and the second simulated eye module 200 according to the commands. The pupillary distance controller 133 is connected to the first rotating module base 131, the second rotating module base 132, and the drive module 140.

[0095] The drive module 140 is the host computer for the first pitch drive component 111, the first rotation drive component 121, the interpupillary distance controller 133, the second pitch drive component 211, and the second rotation drive component 221, and the slave computer for the drive control module 150. It is used to acquire command programs or data from the drive control module 150 and control the corresponding modules or components to execute corresponding instructions. The drive module 140 is connected to the first pitch drive component 111, the first rotation drive component 121, the interpupillary distance controller 133, the second pitch drive component 211, the second rotation drive component 221, and the drive control module 150, respectively.

[0096] The drive control module 150 serves as both the host computer of the drive module 140 and the slave computer of the main control module 160. It acquires command programs or data transmitted from the main control module 160 and controls the drive module 140 to drive the pitch drive component, the rotation module 120, and the interpupillary distance control module 130 to execute corresponding commands. The drive control module 150 is connected to the simulated sclera 102, the human eyeglass lens 104, the drive module 140, and the main control module 160, respectively.

[0097] The main control module 160 is used to control the drive control module 150, acquire and display images transmitted by the image sensor 105 in the first simulated eye module 100 and the second simulated eye module 200, and perform angle compensation on the first simulated eye module 100 and the second simulated eye module 200 based on motion data transmitted by the attitude sensor 106 in the first simulated eye module 100 and the second simulated eye module 200. The main control module 160 is connected to the drive control module 150, the image sensor 105 in the first simulated eye module 100, and the attitude sensor 106 in the second simulated eye module 200.

[0098] The second pitch module 210 is used to acquire instructions from the drive module 140 and provide pitch motion for the second simulated eye module 200. The second pitch module 210 consists of a second pitch drive component 211 and a second pitch transmission component 212.

[0099] The second pitch drive component 211 is connected to the second pitch module bracket 213. The second pitch drive component 211 is connected to the second pitch transmission component 212, and the connection method can be a key connection or a coupling connection.

[0100] The second pitch transmission component 212 can be a synchronous belt pulley assembly, a chain and sprocket assembly, or a V-belt pulley assembly. The second pitch transmission component 212 is connected to the simulated sclera 103 in the second simulated eyeball module 200, and the connection method can be a coupling connection or a key connection. Preferably, the distance between the central axis of the input wheel and the central axis of the output wheel of the second pitch transmission component 212 is 16cm to 30cm.

[0101] The second pitch module bracket 213 is connected to the second pitch drive component 211 and the second rotation transmission component 222, so that the second simulated eyeball module 200 rotates around the rotation axis and the pitch axis.

[0102] The second rotation module 220 is used to receive instructions from the drive module 140 and provide rotational motion for the second simulated eyeball module 200. The second rotation module 220 consists of a second rotation drive component 221 and a second rotation transmission component 222.

[0103] The second rotary drive component 221 is connected to the second rotary module base 132. The second rotary drive component 221 is connected to the second rotary transmission component 222, and the connection method can be a key connection or a coupling connection.

[0104] The second rotary transmission component 222 can be a synchronous belt pulley assembly, a chain and sprocket assembly, or a V-belt pulley assembly. The second rotary transmission component 222 is connected to the second pitch module bracket 213, and the connection method can be a flange connection, welding, or integral with the second pitch module bracket 213, allowing the second simulated eyeball module 200 to rotate around its rotation axis. Preferably, the distance between the central axis of the input wheel and the central axis of the output wheel of the second rotary transmission component 222 is 16cm to 30cm.

[0105] In this embodiment, the VR / AR device eye-tracking testing device can perform vertical pitch, horizontal rotation, pupillary distance changes, and image acquisition and processing on two simulated eye modules. It can accurately simulate various real-world eye movements at different pupillary distances, such as fixation, saccades, and tracking, and process and display the images acquired during the movements according to the characteristics of the simulated human eye. Furthermore, this embodiment reduces the device size through pitch and rotation transmission components, making it easier to operate and use in eye-tracking testing of small-sized VR / AR devices, thus providing greater portability for VR / AR device eye-tracking testing.

[0106] Embodiments 1 and 2 of this invention provide a method for eye-tracking testing of VR / AR devices, such as... Figure 4 As shown. The method includes:

[0107] Step 1: Place the VR / AR device under test in front of the testing device, so that the first simulated eyeball module and the second simulated eyeball module are located inside the pupil box of the VR / AR device under test.

[0108] Step 2: Set the VR / AR device eye-tracking test task type and test trajectory in the main control module. According to the eye-tracking test task type, set the image sensor frame rate, initial rotation position, initial pitch position, maximum rotation angle, maximum pitch angle, maximum rotation angular velocity, maximum pitch angular velocity, pupil center distance p, gaze time, and other test parameters.

[0109] It should be noted that the test trajectory can be trajectory data from eye-tracking tests of real-person VR / AR devices, or it can be trajectory data generated by a computer that includes one or more human eye movements such as gaze, saccade, and following, so that the testing device can realistically simulate the eye movements of a real person when performing eye-tracking tests of VR / AR devices.

[0110] Step 3: Reset the test device. The main control module sends motion data of each module to the drive control module, so that the drive module drives the corresponding modules in the test device to move.

[0111] It should be noted that while performing motion control, the drive control module can adjust the focus of the human eye lens in the first and second simulated eye modules according to scene changes and the current spatial gaze point position to clearly display the object to be focused on, and adjust the pupil size of the simulated iris in the first and second simulated eye modules to simulate the human eye adjusting the intensity of light entering the eye according to the intensity of external light.

[0112] Step 4: The main control module synchronously displays the images captured by the first and second simulated eye modules at the current moment, the spatial gaze point coordinates G(x,y,z) at the current moment, and the gaze point coordinates given by the eye tracking device in the VR / AR device.

[0113] It should be noted that the timing synchronization of the captured image display, the current spatial gaze point coordinate information display, and the gaze point coordinate display in the eye-tracking device of the VR / AR device can be achieved through hardware-level synchronization, such as using synchronization signals or clock signals generated by the drive control module, or through software-level synchronization, such as using a clock synchronization program in the main control module or using network time synchronization.

[0114] Step 5: Repeat steps 2 through 4 until all test tasks are completed. Based on the comparison information of all gaze point coordinates, provide VR / AR eye tracking test results, such as viewpoint accuracy, latency, and gaze point mean square error.

[0115] This invention also provides a control method for a VR / AR device eye-tracking testing apparatus, which can be used to control the first simulated eyeball module and the second simulated eyeball module of the aforementioned eye-tracking testing apparatus to accurately simulate human eye movements, such as... Figure 5 As shown. The method includes:

[0116] Step 1: The attitude sensor acquires the angle data of the first and second simulated eye modules after the test task and test trajectory are set. The main control module compares the current angle data with the set initial angle data and sends a command to the drive control module to reset the first pitch module, the first rotation module, the second pitch module and the second rotation module.

[0117] Step 2: The main control module sends a command to the drive control module according to the pupil center distance set in the test task, controls the pupil distance control module to adjust the pupil center distance of the two simulated eyeball modules, and according to the set VR / AR device eye tracking test trajectory, the trajectory is divided into motion data of the first pitch module, the first rotation module, the second pitch module, and the second rotation module in time sequence.

[0118] It should be noted that the time series can be derived from trajectory data of eye-tracking tests of real-person VR / AR devices, or from computer-generated trajectory data containing one or more human eye movements such as fixation, saccades, and following. It can also be derived from hardware-level synchronization signals, such as synchronization signals or clock signals generated by the drive control module, or from software-level synchronization programs, such as using a clock synchronization program in the main control module or using network time synchronization.

[0119] Step 3: The main control module sends motion data to the drive control module in sequence, so that the corresponding module drives the first simulated eye module and the second simulated eye module to perform pitch or rotation movements.

[0120] In this step, by establishing a spatial rectangular coordinate system, the coordinates of the three-dimensional gaze point at a certain moment in the test trajectory can be transformed into the rotation and pitch angles of the first and second simulated eye modules. For example, as... Figure 6As shown, let A be the pupil center of the first simulated eye module and B be the pupil center of the second simulated eye module. Then AB = p, where p is the set distance between the pupil centers. Let the initial rotation angle of the first simulated eye module be α0, the initial rotation angle of the second simulated eye module be β0, and the initial pitch angle of the simulated eye module be γ0. Taking the midpoint of the line connecting AB as the origin O of the spatial rectangular coordinate system, and the line connecting the two points AB as the x-axis, a spatial rectangular coordinate system can be established using the right-hand screw. The coordinates of the two pupil centers A(0.5p,0,0) and B(-0.5p,0,0) can be obtained. Based on the target gaze point G(x,y,z) at the current moment and the set interpupillary distance p, the required rotation angles α = arctan((x-0.5p) / z)-α0, β = arctan((x+0.5p) / z)-β0, and pitch angles γ = arctan(y / z)-γ0 of the first simulated eye module can be calculated using trigonometric functions.

[0121] It should be noted that the timing transmission of motion data is synchronized with the timing display of the captured image, actual spatial gaze point coordinates, and gaze point coordinates in the VR / AR device eye-tracking test device control method by the main control device. For example, when the main control module acquires and displays the captured image, actual spatial gaze point coordinates, and gaze point coordinates in the VR / AR device eye-tracking device at time i, it simultaneously sends the motion data at time i+1 to the drive control module.

[0122] Step 4: The attitude sensor detects the pitch and rotation angles of the first and second simulated eye modules. The main control module compares the current rotation and pitch angles with the required rotation and pitch angles, so that the drive control module controls the corresponding modules to perform angle compensation.

[0123] Step 5: Repeat steps 3 and 4 until the test trajectory is completed.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the present invention as described above; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An eye-tracking testing device for VR / AR devices, characterized in that, include: The simulated eye module is used to simulate the movement of the human eye and is equipped with an attitude sensor to detect the pitch and rotation angles of the simulated eye module in real time. A pitch module, connected to the simulated eye module, is used to drive the simulated eye module to achieve pitch movement; A rotation module, connected to the simulated eyeball module, is used to drive the simulated eyeball module to achieve rotational movement; The interpupillary distance control module is used to adjust the simulated eye module to simulate the interpupillary distance of different users; The drive module is connected to the pitch module, rotation module and interpupillary distance control module respectively, and is used to provide power to perform corresponding motion control. The drive control module is connected to both the attitude sensor and the drive module. It receives pitch and rotation angle information from the simulated eye module detected by the attitude sensor and controls the drive module to perform error compensation based on this information. The main control module is connected to the drive control module, the attitude sensor, and the VR / AR device eye-tracking system. It sets the test task type and test trajectory, receives the gaze point given by the VR / AR device eye-tracking system, and compares the gaze point currently obtained by the simulated eye module through the VR / AR device eye-tracking system with the set gaze point to achieve accuracy analysis of the VR / AR device eye-tracking system. The simulated eye module includes a first simulated eye module (100) and a second simulated eye module (200). Both the first and second simulated eye modules (100 and 200) are composed of a simulated cornea (101), a simulated iris (102), a simulated sclera (103), a human eye lens (104), an image sensor (105), and a posture sensor (106). The simulated cornea (101), simulated iris (102), human eye lens (104), and image sensor (105) form a coaxial optical system with the optical axis of the human eye lens as its axis. The first and second simulated eye modules (100 and 200) can reflect the light generated by the light source of the eye-tracking device in the VR / AR device, generating a Pulcyn spot for the eye-tracking device to identify and obtain eye-tracking data. The system receives instructions from the drive control module (150) and adjusts the pupil diameter and focal length according to the test task type and scene changes, simulating the human eye adjusting the light intensity entering the eye according to the intensity of external light, and facilitating the eye tracking device in VR / AR devices to recognize the pupil; the image sensor (105) clearly images the object located at the spatial gaze point coordinates at the current moment, and synchronously transmits the image collected during movement to the main control module (160) for display, simulating the human eye retina's acquisition, processing and transmission of light signals; the posture sensor (106) synchronously measures the motion data of the first simulated eye module (100) or the second simulated eye module (200) and synchronously transmits it to the main control module (160) to perform angle compensation for the synchronous movement of the test device, so as to accurately simulate the human eyes viewing the environment and different types of actual movements.

2. The eye-tracking testing device for VR / AR devices according to claim 1, characterized in that, The pitch module includes a first pitch module and a second pitch module; wherein, the first pitch module (110) includes a first pitch drive component (111), a first pitch transmission component (112), and a first pitch module bracket (113) connecting the first pitch drive component (111) and the first pitch transmission component (112); the second pitch module (210) includes a second pitch drive component (211), a second pitch transmission component (212), and a second pitch module bracket (213) connecting the second pitch drive component (211) and the second pitch transmission component (212), wherein the first pitch module (110) is used to obtain instructions from the drive module to drive the first simulated eyeball module to perform pitch motion around the pitch axis, simulating the actual movement of a person's right eye fixation, saccade, and tracking; the second pitch module (210) is used to obtain instructions from the drive module to drive the second simulated eyeball module to perform pitch motion around the pitch axis, simulating the actual movement of a person's left eye fixation, saccade, and tracking. The rotation module includes a first rotation module (120) and a second rotation module (220). The first rotation module (120) consists of a first rotation drive component (121) and a first rotation transmission component (122), and is used to receive instructions from the drive module to drive the first simulated eyeball module (100) to rotate around the rotation axis, simulating the actual movement of a human right eye. The second rotation module (220) consists of a second rotation drive component (221) and a second rotation transmission component (222), and is used to receive instructions from the drive module to drive the second simulated eyeball module (200) to rotate around the rotation axis, simulating the actual movement of a human left eye. The interpupillary distance control module (130) includes a first rotating module base (131), a second rotating module base (132), and an interpupillary distance controller (133). The first rotating module base (131) is connected to the first rotating drive component (121), and the second rotating module base (132) is connected to the second rotating drive component (221). The interpupillary distance controller (133) is connected to the first rotating module base (131), the second rotating module base (132), and the drive module (140) respectively. It is used to adjust the pupil center distance of the first simulated eyeball module (100) and the second simulated eyeball module (200) according to the command or data of the drive module to simulate the interpupillary distance of different people, with a variation range of 40mm to 80mm.

3. The eye-tracking testing device for VR / AR devices according to claim 2, characterized in that, The first pitch drive component (111), the first rotation drive component (121), the second pitch drive component (211), and the second rotation drive component (221) are DC motors, stepper motors, or servo motors; the first pitch transmission component (112), the first rotation transmission component (122), the second pitch transmission component (212), and the second rotation transmission component (222) are gear transmissions, worm gear mechanisms, shafts, synchronous pulley sets, chain sprocket sets, or V-belt pulley sets.

4. The eye-tracking testing device for VR / AR devices according to claim 2, characterized in that, The image sensor (105) is located in the posterior hemisphere of the simulated sclera (103) and in the pupil box of the human eyeglass lens (104); the simulated iris (102) is a non-white aperture with an adjustment structure; the human eyeglass lens (104) has a focal length adjustment module and at least one positive power lens.

5. The eye-tracking testing device for VR / AR devices according to claim 2, characterized in that, The simulated cornea (101) is a convex or concave lens made of glass, transparent resin, or silicone gel.

6. The eye-tracking testing device for VR / AR devices according to claim 2, characterized in that, The simulated iris (102) is a variable aperture with a mechanical adjustment structure, or a variable aperture with an electric adjustment mechanism, or a transmissive display, or a transmissive LCD display, or a transmissive LED display, or a transmissive OLED display.

7. The eye-tracking testing device for VR / AR devices according to claim 1, characterized in that, The drive module (140) is used to acquire command programs or data from the drive control module (150) and control the corresponding modules or components to execute corresponding instructions; the drive control module (150) is used to acquire command programs or data transmitted by the main control module (160), synchronously control the drive module (140) to drive the corresponding modules or components to execute instructions, and synchronously control the human eye lens (104) and the simulated sclera (103) in the first simulated eye module (100) and the second simulated eye module (200) to adjust the focal length and pupil size according to the test task and scene changes; at the same time, it synchronizes the timing information with the main control module (160) and synchronously sends the current spatial gaze point coordinate information to the main control module (160); the main control module (160) is used to decompose the test trajectory into the motion of each module. The data is sent to the drive control module (150) to control each module. At the same time, the images input by the image sensors (105) in the first simulated eye module (100) and the second simulated eye module (200) are edited according to the test task to simulate the sensitivity of different human eyes to color and brightness information in the environment. Simultaneously, the images transmitted by the image sensors (105) in the first simulated eye module (100) and the second simulated eye module (200), the spatial gaze point coordinates at the current time and the spatial gaze point coordinates given by the eye tracking device of the VR / AR device are acquired and displayed. According to the motion data transmitted by the posture sensor (106), the drive control module (150) is controlled to perform angle compensation on the first simulated eye module (100) and the second simulated eye module (200).

8. A testing method for eye tracking in VR / AR devices, characterized in that, Includes the following steps: S1. Place the VR / AR device under test in front of the testing apparatus according to any one of claims 1 to 7. The simulated eye module is located inside the pupil box of the VR / AR device under test; S2. Set the eye-tracking test task type and test trajectory for VR / AR devices through the main control module; S3. Drive the pitch and rotation modules to make the simulated eye module pitch and rotate according to the set test trajectory; S4. The pitch and rotation angles of the simulated eye module are detected in real time using an attitude sensor, and error compensation is performed through a drive control module; S5. The main control module receives gaze point information from the eye-tracking system of the VR / AR device based on the simulated eye movement module. S6. Compare the gaze point of the current simulated eye module with the gaze point given by the eye-tracking system of the VR / AR device, and analyze the accuracy of the eye-tracking system of the VR / AR device.

9. The testing method for eye tracking in VR / AR devices according to claim 8, characterized in that, It also includes the following steps: adjusting the interpupillary distance of the simulated eye module through the interpupillary distance control module according to the test requirements, so as to simulate the eye characteristics of different users for testing; The image acquisition module is used to collect image information of the simulated eye module in the VR / AR environment in real time, so as to further analyze the performance of the eye tracking system of VR / AR device.

10. The testing method for eye tracking in VR / AR devices according to claim 8, characterized in that, Step S4 specifically includes: S4.1 The attitude sensor (106) acquires the angle data of the first simulated eye module (100) and the second simulated eye module (200) after the test task and test trajectory are set. The main control module (160) compares the current angle data with the set initial angle data and controls the drive control module (150) to reset the corresponding module. S4.2 The main control module (160) sends a command to the drive control module (150) according to the pupil center distance set in the test task, controls the pupil distance control module (130) to adjust the pupil center distance of the two simulated eyeball modules, and according to the set VR / AR device eye tracking test trajectory, the trajectory is divided into motion data of each module in time sequence; S4.3 The main control module (160) sends motion data to the drive control module (150) in sequence, so that the corresponding module drives the first simulated eye module (100) and the second simulated eye module (200) to perform pitch or rotation. S4.4 The attitude sensor (106) detects the pitch and rotation angles of the first simulated eye module (100) and the second simulated eye module (200). The main control module (160) compares the current rotation and pitch angles with the required rotation and pitch angles, so that the drive control module (150) controls the corresponding modules to perform angle compensation. S4.5 repeats until the test trajectory is completed.

Images