Eye movement tracking test device and test method for VR / AR equipment

Abstract

The invention discloses an eye movement tracking test device and method for VR / AR equipment. The eye movement tracking test device comprises an eyeball simulation module, a pitching module, a rotating module, an interpupillary distance control module, a driving module, a driving control module and a main control module. The main control module sets a test task type and a test track, the rotating module and the pitching module drive the simulation eyeball module to realize rotating motion and pitching motion, and the attitude sensor and the driving control module detect the pitching angle and the rotating angle of the simulation eyeball module and control the corresponding modules to perform error compensation. And the main control module compares the current staring point with a staring point given by the VR / AR equipment eye movement tracking system to realize precision analysis of the VR / AR equipment eye movement tracking system. The VR / AR equipment eye movement tracking test device effectively simulates human eye movement, solves the problem that an existing test device lacks actual movement image information, visually and accurately reflects various real movement of human eyes, and improves the stability of VR / AR equipment eye movement tracking test.

Description

Technical Field

[0001] The present invention relates to the field of visual equipment technology, and more specifically, to an eye tracking test device and a test method for VR / AR equipment. Background Art

[0002] With the rapid development of fields such as intelligent robots and display devices, the virtual reality (VR) and augmented reality (AR) industries have received more and more attention. This type of near-eye display product is a head-mounted device. The distance between the human eye and the display module is relatively close. It uses optical technology to realize virtual and enlarged images in the user's near-eye range to provide users with intuitive images, videos or text information. Nowadays, related applications have gradually been applied to industries such as film and television, games, and medical care. The eye tracking device is an important component of near-eye display products such as VR / AR. It is designed to provide users with a more immersive interactive experience. The device converts eye movements into a data stream, which contains information such as pupil position, gaze vector and gaze point of each eye. In essence, the eye tracking device is a decoder of eye movements, so a large number of eye tracking test tasks need to be performed on the device to improve the decoding speed and accuracy of eye movements.

[0003] The traditional VR / AR device eye tracking test method requires the subject to first fix the head with a headrest, then wear a VR / AR device with an eye tracking device, and then display a calibration point on the screen at a certain distance from the human eye to prompt the eye movement so that the gaze point position matches the calibration point position, thereby realizing the VR / AR device eye tracking device test. However, the traditional eye tracking test method is time-consuming, causes visual fatigue to the subject, and affects the accuracy of the acquired data. At the same time, this test method requires recruiting a large number of different subjects to participate in the test to improve the robustness of the gaze tracking device. To solve this problem, a learning-based eye tracking test method has been proposed. However, this method requires a large number of data sets with pupil center coordinates, gaze vectors or gaze points as labels and static eye photos as features to train and verify the model, so it cannot intuitively and accurately reflect the various actual movements of the human eye, such as gaze, eye saccades, and tracking.

[0004] Patent document CN117991502A proposes an eye tracking device and test device. The device connects a simulated eyeball to a first rotating component, and makes the line connecting the pupil and the center of the simulated eyeball form a certain angle with the rotation axis of the first rotating component. At the same time, the second rotating component is connected to the first rotating component, and the pitch angle, horizontal rotation angle and angle generated by the two rotating components are used to simulate the movement of the human eye. However, the device does not take into account the image information obtained by the human eye in actual movement, so there is still a big difference between it and the real human eye. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and provide an eye tracking test device and test method for VR / AR devices that can simulate the human eye to obtain image information in actual motion and simultaneously simulate various actual motions of the human eye such as fixation, eye saccade and tracking. Through the pitch and rotation motion of two simulated eyeball modules, changes in pupil distance, and image acquisition and processing, it is possible to accurately simulate various actual motions of human eyes with different pupil distances such as fixation, eye saccade and tracking, and synchronously simulate the images obtained by the human eye in these motions, and at the same time, process and display the images obtained during motion according to the simulated human eye characteristics, effectively solving the problem that the eye tracking test device for VR / AR devices lacks image information obtained by the human eye in actual motion. Through the pitch transmission component and the rotation transmission component, the device size is reduced, making the present invention easier to operate and use in the eye tracking test of small-volume VR / AR devices, providing stronger portability for the eye tracking test of VR / AR devices, and effectively solving the problem of large size and low portability of the eye tracking test device for VR / AR devices.

[0006] The technical solution of the present invention is as follows:

[0007] On the one hand, the present invention provides an eye tracking test device for VR / AR equipment, which is characterized by comprising:

[0008] The simulated eyeball module is used to simulate the movement of the human eye and is equipped with a posture sensor for detecting the pitch angle and rotation angle of the simulated eyeball module in real time;

[0009] A pitch module, connected to the simulated eye module, and used to drive the simulated eye module to achieve pitch movement;

[0010] A rotation module, connected to the simulated eyeball module, and used to drive the simulated eyeball module to achieve rotational motion;

[0011] A pupil distance control module, used for adjusting the simulated eyeball module to simulate the pupil distance of different users;

[0012] A driving module, connected to the pitch module, the rotation module and the pupil distance control module respectively, for providing power to perform corresponding motion control;

[0013] A drive control module, connected to the attitude sensor and the drive module respectively, for receiving pitch angle and rotation angle information of the simulated eyeball module detected by the attitude sensor, and controlling the drive module to perform error compensation based on the information;

[0014] The main control module is respectively connected to the drive control module, the posture sensor and the VR / AR device eye tracking system, and is used to set the test task type and the test trajectory, receive the gaze point given by the VR / AR device eye tracking system, and compare the gaze point currently obtained by the simulated eye module through the VR / AR device eye tracking system with the set gaze point, so as to achieve accuracy analysis of the VR / AR device eye tracking system.

[0015] Furthermore, the simulated eyeball module comprises: a first simulated eyeball module and a second simulated eyeball module, wherein the first simulated eyeball module and the second simulated eyeball module are both composed of a simulated cornea, a simulated iris, a simulated sclera, a human eye lens, an image sensor and a posture sensor, wherein the simulated cornea, the simulated iris, the human eye lens and the image sensor form a coaxial optical system with the optical axis of the human eye lens as the axis; the first simulated eyeball module and the second simulated eyeball module can reflect the light generated by the light source of the eye tracking device in the VR / AR device, generate a Purkinje light spot for the eye tracking device to identify the light spot to obtain eye movement data, and receive instructions from the drive control module, and according to the test task type The pupil diameter and focal length are adjusted according to the type and scene changes, simulating the human eye to adjust the light intensity entering the eye according to the intensity of external light, and facilitating the eye tracking device in the VR / AR device to identify the pupil; the image sensor clearly images the object located at the coordinates of the gaze point in space at the current moment, and synchronously transmits the image collected during movement to the main control module for display, simulating the collection, processing and transmission of light signals by the human eye retina; the posture sensor synchronously measures the movement data of the first simulated eyeball module or the second simulated eyeball module and synchronously transmits it to the main control module to synchronously perform angle compensation for the movement of the test device, so as to accurately simulate the viewing environment of human binoculars and different types of actual movements;

[0016] The pitch module comprises a first pitch module and a second pitch module; wherein the first pitch module comprises a first pitch driving component, a first rotating transmission component and a first pitch module bracket connecting the first pitch driving component and the first rotating transmission component; the second pitch module comprises a second pitch driving component, a second pitch transmission component and a second pitch module bracket connecting the second pitch driving component and the second rotating transmission component, the first pitch module is used to obtain instructions from the driving module, drive the first simulated eyeball module to perform pitch movement around the pitch axis, and simulate the actual movement of the right eye of a person to gaze, saccade and follow; the second pitch module is used to obtain instructions from the driving module, drive the second simulated eyeball module to perform pitch movement around the pitch axis, and simulate the actual movement of the left eye to gaze, saccade and follow;

[0017] The rotation module comprises a first rotation module and a second rotation module, wherein the first rotation module is composed of a first rotation driving component and a first rotation transmission component, and is used to obtain instructions from the driving module, drive the first simulated eyeball module to rotate around the rotation axis, and simulate the actual movement of the right eye of a person; the second rotation module is composed of a second rotation driving component and a second rotation transmission component, and is used to obtain instructions from the driving module, drive the second simulated eyeball module to rotate around the rotation axis, and simulate the actual movement of the left eye of a person;

[0018] The pupil distance control module includes a first rotating module base, a second rotating module base and a pupil distance controller, wherein the first rotating module base is connected to the first rotating driving component, the second rotating module base is connected to the second rotating driving component, and the pupil distance controller is respectively connected to the first rotating module base, the second rotating module base and the driving module, and is used to adjust the pupil center distance of the first simulated eyeball module and the second simulated eyeball module according to the command or data of the driving module to simulate the pupil distance of different people, with a variation range of 40 mm to 80 mm.

[0019] Further, the first pitch drive component, the first rotation drive component, the second pitch drive component and the second rotation drive component are DC motors, stepper motors or servo motors; the first pitch transmission component, the first rotation transmission component, the second pitch transmission component and the second rotation transmission component are gear transmissions, worm gear mechanisms, shafts, synchronous pulley sets, chain sprocket sets or V-belt pulley sets.

[0020] The image sensor is located in the rear hemisphere of the simulated sclera and in the pupil box of the human eyeglasses; the simulated iris is a stop with a non-white surface and an adjustment structure; the human eye lens has a focal length adjustment module and at least one positive focal power lens.

[0021] The simulated cornea is a convex-concave lens made of glass, transparent resin, or silicone gel.

[0022] The simulated iris is a variable iris with a mechanical adjustment structure, or a variable iris with an electric adjustment mechanism, or a transmissive display, or a transmissive LCD display, or a transmissive LED display, or a transmissive OLED display.

[0023] Furthermore, the driving module is used to obtain the command program or data from the driving control module, and control the corresponding modules or components to execute the corresponding instructions; the driving control module is used to obtain the command program or data transmitted by the main control module, synchronously control the driving module to drive the corresponding modules or components to execute instructions, and synchronously control the human eye lens and the simulated sclera in the first simulated eyeball module and the second simulated eyeball module to adjust the focal length and pupil size according to the test tasks and scene changes; at the same time, synchronize the timing information with the main control module, and synchronously send the current moment space gaze point coordinate information to the main control module; the main control module is used to decompose the test trajectory into motion data of each module , and synchronously sends instructions to the drive control module to control each module, and at the same time, edits and processes the images input by the image sensors in the first simulated eyeball module and the second simulated eyeball module according to the test task to simulate the sensitivity of different human eyes to information such as color and brightness in the environment, and synchronously obtains and displays the images transmitted by the image sensors in the first simulated eyeball module and the second simulated eyeball module, the spatial gaze point coordinates at the current moment, and the spatial gaze point coordinates given by the eye tracking device of the VR / AR device, and synchronously controls the drive control module to perform angle compensation on the first simulated eyeball module and the second simulated eyeball module according to the motion data transmitted by the posture sensor.

[0024] In a second aspect, the present invention provides a testing method for eye tracking of VR / AR devices, which is characterized in that it includes the following steps:

[0025] S1. Place the VR / AR device to be tested in front of the test device according to any one of claims 1 to 7, so that the simulated eyeball module is located in the pupil box of the VR / AR device to be tested;

[0026] S2. Set the VR / AR device eye tracking test task type and test trajectory through the main control module;

[0027] S3 drives the pitch module and the rotation module to make the simulated eyeball module perform pitch and rotation movements according to the set test trajectory;

[0028] S4. Use the attitude sensor to detect the pitch angle and rotation angle of the simulated eyeball module in real time, and perform error compensation through the drive control module;

[0029] S5. The main control module receives the gaze point information given by the VR / AR device eye tracking system based on the simulated eyeball module movement;

[0030] S6. Compare the gaze point of the current simulated eyeball module with the gaze point given by the eye tracking system of the VR / AR device, and analyze the accuracy of the eye tracking system of the VR / AR device.

[0031] Furthermore, the method further includes the steps of: adjusting the pupil distance of the simulated eyeball module through the pupil distance control module according to the test requirements to simulate the eye characteristics of different users for testing;

[0032] The image acquisition module is used to collect image information of the simulated eye module in the VR / AR environment in real time to further analyze the performance of the VR / AR device eye tracking system.

[0033] Further, the step S4. specifically comprises:

[0034] S4.1 The posture sensor obtains the angle data of the first simulated eyeball module and the second simulated eyeball module after the test task and the test trajectory are set. The main control module compares the current angle data with the set initial angle data and controls the drive control module to reset the corresponding module;

[0035] The S4.2 main control module sends instructions to the drive control module according to the pupil center distance set in the test task, controls the pupil distance control module to adjust the pupil center distance of the two simulated eyeball modules, and splits the trajectory into motion data of each module in time series according to the set VR / AR device eye tracking test trajectory.

[0036] S4.3 The main control module sends the motion data to the drive control module in a timely manner, so that the corresponding module drives the first simulated eyeball module and the second simulated eyeball module to perform pitch motion or rotation motion;

[0037] S4.4 The attitude sensor detects the current pitch angle and rotation angle of the first simulated eyeball module and the second simulated eyeball module, and the main control module compares the current rotation angle and pitch angle with the required rotation angle and pitch angle, so that the drive control module controls the corresponding module to perform angle compensation;

[0038] S4.5 is repeated in a loop until the test track is completed.

[0039] Compared with the prior art, the present invention has the following obvious outstanding substantive features and significant advantages:

[0040] 1. The present invention utilizes an eyeball simulation module, a pitch module, a rotation module and an interpupillary distance control module, and interconnects the modules to simulate actual human eye movements such as fixation, saccade and tracking at different interpupillary distances, and obtains images collected when the eyeball simulation module moves, three-dimensional coordinate information of the spatial gaze point and three-dimensional coordinate information of the spatial gaze point given by the VR / AR device in real time and synchronously, thereby effectively solving the problem that the VR / AR device eye movement tracking test device lacks image information obtained by the human eye in actual movement.

[0041] 2. The present invention obtains the pitch angle or rotation angle of the two simulated eye modules through the posture sensor, and uses the main control module and the drive control module to perform angle compensation, thereby realizing accurate simulation of the actual movements of the human eye such as gaze, eye saccade and following. At the same time, the pitch transmission components in the pitch module and the rotation transmission components in the rotation module are used to reduce the size of the device, effectively solving the problems of large size and low portability of the VR / AR device eye tracking test device.

[0042] 3. The test method of the present invention can perform eye tracking tests on VR / AR devices according to different test requirements. The method can initialize eye movement parameters such as starting angle, ending angle, movement speed, movement acceleration, uniform movement time, accelerated movement time, etc. according to different test task types and test trajectories, and obtain the three-dimensional coordinates of the current space gaze point, and compare them with the gaze point coordinates given by the eye tracking system in the VR / AR device to achieve accuracy analysis of the VR / AR device eye tracking system. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Attached Figure 1 This is a structural schematic diagram of Embodiment 1 of the eye tracking test device and test method for VR / AR equipment of the present invention;

[0044] Attached Figure 2 It is a structural schematic diagram of the eyeball simulation module of Embodiment 1 and Embodiment 2 of the eye tracking test device and test method for VR / AR equipment of the present invention;

[0045] Attached Figure 3 This is a structural schematic diagram of Embodiment 2 of the eye tracking test device and test method for VR / AR equipment of the present invention;

[0046] Attached Figure 4 An overview flow chart of an eye tracking test method for VR / AR devices provided in Embodiments 1 and 2 of the present invention;

[0047] Attached Figure 5 A flowchart of an overview of a control method for an eye tracking test device for a VR / AR device provided in Embodiments 1 and 2 of the present invention.

[0048] Attached Figure 6 A schematic diagram of an overview of an eye tracking test method for a VR / AR device provided in Embodiment 1 and Embodiment 2 of the present invention.

[0049] In the figure: 100 is a first simulated eyeball module, 101 is a simulated cornea, 102 is a simulated iris, 103 is a simulated sclera, 104 is a human eye lens, 105 is an image sensor, 110 is a first pitch module, 111 is a first pitch driving component, 112 is a first pitch transmission component, 113 is a first pitch module bracket, 120 is a first rotation module, 121 is a first rotation driving component, 122 is a first rotation transmission component, 130 is a pupil distance control module, 131 is a first rotation Module base, 132 is the second rotating module base, 133 is the pupil distance controller, 140 is the driving module, 150 is the driving control module, 160 is the main control module, 200 is the second simulated eyeball module, 210 is the second pitch module, 211 is the second pitch driving component, 212 is the second pitch transmission component, 213 is the second pitch module bracket, 220 is the second rotating module, 221 is the second rotating driving component, 222 is the second rotating transmission component, and 300 is the VR / AR device.

[0050] It should be understood that the above drawings are only schematic and are not drawn to scale. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be described in detail below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The above scheme is further described below in conjunction with specific implementation examples. The preferred embodiments of the present invention are described in detail as follows: Embodiment 1

[0053] See also Figure 1 , Figure 1 The present invention is a structural schematic diagram of embodiment 1 of the eye tracking test device and test method for VR / AR equipment, comprising a first simulated eyeball module 100, a first pitch module 110, a first pitch driving component 111, a first pitch transmission component 112, a first pitch module bracket 113, a first rotation module 120, a first rotation driving component 121, a first rotation transmission component 122, a pupil distance control module 130, a first rotation module base 131, a second rotation module base 132, a drive module 140, a drive control module 150, a main control module 160, a second simulated eyeball module 200, a second pitch module 210, a second pitch driving component 211, a second pitch transmission component 212, a second pitch module bracket 213, a second rotation module 220, a second rotation driving component 221, and a second rotation transmission component 222.

[0054] The first simulated eyeball module 100 and the second simulated eyeball module 200 are used to simulate the actual movement of the right eye and the left eye of a person, respectively, and synchronously transmit the image data and movement data obtained by the movement to the main control module 160, so as to realize the accurate simulation of the human binocular viewing environment and different types of actual movement. The first simulated eyeball module 100 and the second simulated eyeball module 200 are both composed of a simulated cornea 101, a simulated iris 102, a simulated sclera 103, a human eye lens 104, an image sensor 105 and a posture sensor 106.The first simulated eyeball module 100 and the second simulated eyeball module 200 both have the following six functions: on the one hand, the simulated cornea 101, simulated iris 102, human eye lens 104 and image sensor 105 in the first simulated eyeball module 100 and the second simulated eyeball module 200 form a coaxial optical system, and the optical axis of the system is the optical axis of the human eye lens 104 to simulate the imaging structure in the human eye; on the other hand, the simulated cornea 101 in the first simulated eyeball module 100 and the second simulated eyeball module 200 can reflect the light generated by the light source of the eye tracking device in the VR / AR device to generate a pulse. On the one hand, the simulated iris 102 in the first simulated eyeball module 100 and the second simulated eyeball module 200 can synchronously receive instructions from the drive control module 150, and adjust the pupil diameter of the first simulated eyeball module 100 or the second simulated eyeball module 200 according to the test task type and scene changes, simulating the human eye to adjust the light intensity entering the eye according to the intensity of external light, and facilitating the eye tracking device in the VR / AR device to identify the pupil to obtain the pupil center coordinates and pupil diameter; on the other hand, the first simulated eyeball module 100 and the second simulated eyeball module 200 can synchronously receive instructions from the drive control module 150, and adjust the pupil diameter of the first simulated eyeball module 100 or the second simulated eyeball module 200 according to the test task type and scene changes, so as to simulate the human eye to adjust the light intensity entering the eye according to the intensity of external light, and facilitate .... The human eye lens 104 in the eyeball module 100 and the second simulated eyeball module 200 can synchronously receive the instruction of the driving control module 150, and adjust the focal length of the first simulated eyeball module 100 or the second simulated eyeball module 200 according to the test task type and scene changes, so that the image sensor 105 in the first simulated eyeball module 100 and the second simulated eyeball module 200 can clearly image the object located at the spatial gaze point coordinates at the current moment; on the other hand, the image sensor 105 in the first simulated eyeball module 100 and the second simulated eyeball module 200 can The images collected when the first simulated eyeball module 100 and the second simulated eyeball module 200 move are synchronously transmitted to the main control module 160 for display, simulating the collection, processing and transmission of light signals by the retina of the human eye; Fifth, the posture sensor 106 in the first simulated eyeball module 100 and the second simulated eyeball module 200 can synchronously measure the movement data of the first simulated eyeball module 100 or the second simulated eyeball module 200 before and during the test task, and synchronously transmit the movement data to the main control module 160, and perform angle compensation on the movement synchronization of the VR / AR eye tracking test device according to the test task.Preferably, the maximum pitch angle and maximum rotation angle of the first simulated eyeball module 100 and the second simulated eyeball module 200 are ±45 degrees, and when the first simulated eyeball module 100 and the second simulated eyeball module 200 perform pitch or rotation movements, the maximum angular velocity is 120rpm, and the maximum angular acceleration is 20πrad / s2. Further preferably, the pitch angle range of the first simulated eyeball module 100 and the second simulated eyeball module 200 is ±20 degrees to ±45 degrees, and the rotation angle range is ±20 degrees to ±75 degrees, so that the test device can truly simulate the eyeball movement angle of a normal human eye.

[0055] The first pitch module 110 is used to obtain the instruction of the driving module 140, so that the first simulated eyeball module 100 performs pitch movement around the pitch axis to simulate the actual movement of the human right eye, such as gaze, eye saccade and tracking. The first pitch module 110 is composed of a first pitch driving component 111 and a first pitch transmission component 112. Preferably, the first pitch module 110 also has an error compensation device, such as a photoelectric encoder, a Hall sensor, a magnetoresistive sensor, an inertial sensor, etc., for closed-loop control of the VR / AR eye tracking test device. Further preferably, the first pitch module 110 has a matte black surface coating, which is convenient for the VR / AR device eye tracking device to identify the first simulated eyeball module 100 for eye tracking testing.

[0056] The first pitch driving component 111 may be a DC motor, a stepping motor, or a servo motor. The first pitch driving component 111 is connected to the first pitch transmission component 112 by a key connection or a coupling connection.

[0057] The first pitch transmission component 112 may be a gear transmission, a worm gear mechanism, or a shaft. The first pitch transmission component 112 is connected to the simulated sclera 103 in the first simulated eyeball module 100 by a coupling or a key.

[0058] The first pitch module bracket 113 is used to connect the first pitch driving component 111 and the first rotating transmission component 122, so that the first simulated eyeball module 100 rotates around the rotation axis to simulate the actual movement of the right eye of a person, such as gaze, eye saccade and tracking. The first pitch module bracket 113 can be a plate made of metal, engineering plastic, or 3D printing material. The first pitch module bracket 113 is connected to the first pitch driving component 111, and the connection method can be threaded connection, welding, or integrated with the first pitch driving component 111. The first pitch module bracket 113 is connected to the first rotating transmission component 122, and the connection method can be flange connection, welding, or integrated with the first rotating transmission component 122. Preferably, the first pitch module bracket 113 is an L-shaped plate with a black matte surface coating, which is convenient for the VR / AR device eye tracking device to identify the first simulated eyeball module 100 for eye tracking test.

[0059] The first rotation module 120 is used to obtain the instructions of the driving module 140, provide rotational movement for the first simulated eyeball module 100, and realize the simulation of the actual movement of the human right eye such as gaze, eye saccade and tracking. The first rotation module 120 is composed of a first rotation drive component 121 and a first rotation transmission component 122. Preferably, the first rotation module 120 also has an error compensation device, such as a photoelectric encoder, a Hall sensor, a magnetoresistive sensor, an inertial sensor, etc., for closed-loop control of the VR / AR eye tracking test device. Further preferably, the first rotation module 120 has a matte black surface coating, which facilitates the VR / AR device eye tracking device to identify the first simulated eyeball module 100 for eye tracking testing.

[0060] The first rotating drive component 121 may be a DC motor, a stepping motor, or a servo motor. The first rotating drive component 121 is connected to the first rotating transmission component 122 by a key connection or a coupling connection. The first rotating drive component 121 is connected to the first rotating module base 131 by a threaded connection, welding, or integration with the first rotating module base 131.

[0061] The first rotating transmission component 122 can be a gear transmission, a worm gear mechanism, or a shaft. The first rotating transmission component 122 is connected to the first pitch module bracket 113, and the connection method can be flange connection, welding, or integration with the first rotating transmission component 122, so that the first simulated eyeball module 100 rotates around its rotation axis.

[0062] The pupil distance control module 130 is used to receive commands or data from the driving module 140, and adjust the pupil center distance of the first simulated eyeball module 100 and the second simulated eyeball module 200 according to the commands or data to simulate the pupil distance of different people. The pupil distance control module 130 is composed of a first rotating module base 131, a second rotating module base 132 and a pupil distance controller 133. The pupil distance control module 130 controls the pupil center distance of the first simulated eyeball module 100 and the second simulated eyeball module 200 to change in the range of 40mm to 80mm. Preferably, the pupil distance control module 130 controls the pupil distance to change in the range of 56mm to 64mm to match the pupil distance of a normal human eye. Further preferably, the pupil distance control module 130 also has an error compensation device, such as a photoelectric encoder, a Hall sensor, or a magnetoresistive sensor, or an inertial sensor, and is connected to the main control module 160 for closed-loop control of the VR / AR eye tracking test device. Further preferably, the pupil distance control module 130 also has a black matte surface coating, which facilitates the VR / AR device eye tracking device to identify the first simulated eyeball module 100 and the second simulated eyeball module 200, so as to facilitate eye tracking testing.

[0063] The first rotating module base 131 can be a platform made of metal or engineering plastics, and is used to fix the first rotating drive component 121 on the pupil distance control module 130. The first rotating module base 131 is connected to the first rotating drive component 121, and the connection method can be threaded connection, welding, or integrated with the first rotating drive component 121.

[0064] The second rotating module base 132 can be a platform made of metal or engineering plastics, and is used to fix the second rotating drive component 221 on the pupil distance control module 130. The second rotating module base 132 is connected to the second rotating drive component 221, and the connection method can be threaded connection, welding, or integrated with the second rotating drive component 221.

[0065] The pupil distance controller 133 can be a linear motor, a ball screw, or a translation stage. The pupil distance controller 133 is connected to the first rotation module base 131, and the connection method can be a key connection, a coupling connection, or a threaded connection. The pupil distance controller 133 is connected to the second rotation module base 132, and the connection method can be a key connection, a coupling connection, or a threaded connection. The pupil distance controller 132 is connected to the drive control module 150, and the connection method can be connected through USB, serial port, general I / O, etc.

[0066] The driving module 140 is used to obtain the command program or data from the driving control module 150, and control the corresponding modules or components to execute the corresponding instructions. The driving module 140 is the upper computer of the first pitch driving component 111, the first rotation driving component 121, the pupil distance controller 133, the second pitch driving component 211, and the second rotation driving component 221, and the lower computer of the driving control module 150. The driving module 140 can be a stepper motor driving board, a servo motor driving board, an ARM chip, a DSP chip, an FPGA chip, a single-chip microcomputer, or an industrial computer. The driving module 140 is respectively connected to the first pitch driving component 111, the first rotation driving component 121, the pupil distance controller 133, the second pitch driving component 211, the second rotation driving component 221 and the driving control module 150, and the connection method can be wire welding, crimping, plug-in connection, USB, serial port, parallel port, etc.

[0067] The drive control module 150 is the upper computer of the drive module 140 and the lower computer of the main control module 160, and has a component for generating a synchronization signal. The drive control module has three functions. On the one hand, the drive control module 150 obtains the command program or data transmitted by the main control module 160, and synchronously controls the drive module 140 to drive the corresponding module or component to execute the instruction; on the other hand, the drive control module 150 synchronously controls the human eye lens 104 and the simulated sclera 102 in the first simulated eyeball module 100 and the second simulated eyeball module 200 to adjust the focal length and pupil size according to the test task and scene changes; on the other hand, the drive control module 150 synchronizes the timing information with the main control module 160, and synchronously sends the current moment space gaze point coordinate information to the main control module 160. The drive control module 150 can be a server, an ARM chip, a DSP chip, an FPGA chip, a single-chip microcomputer, an industrial computer, etc. The component that generates the synchronization signal in the drive control module 150 can be a crystal oscillator, a clock chip, or a pulse generator. The drive control module 150 is connected to the simulated sclera 102 and the human eye lens 104 in the first simulated eyeball module 100 and the second simulated eyeball module 200, and the connection method can be a wired connection such as USB, serial port, parallel port, etc., or a wireless connection such as using Bluetooth, local area network, LoRa, etc. The drive control module 150 is connected to the drive module 140, and the connection method can be wire welding, crimping, plug-in connection, etc. The drive control module 150 is connected to the main control module 160, and the connection method can be a wired connection such as USB, serial port, parallel port, etc., or a wireless connection such as using Bluetooth, local area network, LoRa, etc. The timing synchronization between the drive control module 150 and the main control module 160 can be synchronization at the hardware level, such as using the synchronization signal generated by the synchronization signal generating component in the drive control module 140 for synchronization, or synchronization at the software level, such as using a clock synchronization program, using network time synchronization, etc.

[0068] The main control module 160 is the upper computer of the drive control module 150. The main control module has three functions. On the one hand, the main control module 160 decomposes the test trajectory into motion data of each module according to the test task and the test trajectory, and synchronously sends instructions to the drive control module 150 to control each module. At the same time, according to the test task, the image input by the image sensor 105 in the first simulated eyeball module 100 and the second simulated eyeball module 200 is edited and processed, such as changing the brightness, contrast, saturation, gamma value and resolution of the input image, to simulate the sensitivity of different human eyes to information such as color and brightness in the environment; on the other hand, the main control module The control module 160 synchronously acquires and displays the image transmitted by the image sensor 105 in the first simulated eyeball module 100 and the second simulated eyeball module 200, the spatial gaze point coordinates at the current moment, and the spatial gaze point coordinates given by the eye tracking device of the VR / AR device; on the other hand; the main control module 160 synchronously controls the drive control module 150 to perform angle compensation on the first simulated eyeball module 100 and the second simulated eyeball module 200 according to the motion data transmitted by the posture sensor 106 in the first simulated eyeball module 100 and the second simulated eyeball module 200. The main control module 160 can be a personal computer, a server, an ARM chip, a DSP chip, an FPGA chip, a single-chip microcomputer, an industrial computer, etc. The main control module 160 is connected to the image sensor 105 in the first simulated eyeball module 100 and the second simulated eyeball module 200, and the connection method can be connected through video interfaces such as VGA, HDMI, DVI, DisplayPort, USB, serial port, and general I / O. The main control module 160 is connected to the posture sensor 106 in the first simulated eyeball module 100 and the second simulated eyeball module 200, and the connection mode can be a wired connection such as USB, serial port, parallel port, etc., or a wireless connection such as Bluetooth, local area network, LoRa, etc. The main control module 160 is connected to the drive control module 150, and the connection mode can be a wired connection such as USB, serial port, parallel port, etc., or a wireless connection such as Bluetooth, local area network, LoRa, etc.

[0069] The second pitch module 210 is used to obtain the instructions of the driving module 140, provide pitch movement for the second simulated eyeball module 200, and realize the simulation of the actual movement of the human left eye such as gaze, eye saccade and tracking. The second pitch module 210 is composed of a second pitch driving component 211 and a second pitch transmission component 212. Preferably, the second pitch module 210 also has an error compensation device, such as a photoelectric encoder, a Hall sensor, or a magnetoresistive sensor, or an inertial sensor, etc., for closed-loop control of the VR / AR eye tracking test device. Further preferably, the second pitch module 210 has a matte black surface coating, which facilitates the VR / AR device eye tracking device to identify the second simulated eyeball module 200 for eye tracking testing.

[0070] The second pitch driving component 211 may be a DC motor, a stepping motor, or a servo motor. The second pitch driving component 211 is connected to the second pitch transmission component 212 by a key connection or a coupling connection. The second pitch driving component 211 is connected to the second pitch module bracket 213 by a threaded connection, a welding connection, or an integration with the second pitch module bracket 213.

[0071] The second pitch transmission component 212 can be a gear transmission, a worm gear mechanism, or a shaft. The second pitch transmission component 212 is connected to the simulated sclera 103 in the second simulated eyeball module 200 by a coupling or a key.

[0072] The second pitch module bracket 213 is used to connect the second pitch driving component 211 and the second rotating transmission component 222, so that the second simulated eyeball module 200 rotates around the rotation axis to simulate various actual movements of the human eye, such as gaze, eye saccade and tracking. The second pitch module bracket 213 can be a plate made of metal, engineering plastic, or 3D printing material. The second pitch module bracket 213 is connected to the second pitch driving component 211, and the connection method can be threaded connection, welding, or integrated with the second pitch driving component 211. The second pitch module bracket 213 is connected to the second rotating transmission component 222, and the connection method can be flange connection, welding, or integrated with the second rotating transmission component 222. Preferably, the second pitch module bracket 213 is an L-shaped plate with a black matte surface coating, which is convenient for the VR / AR device eye tracking device to identify the second simulated eyeball module 200 for eye tracking testing.

[0073] The second rotation module 220 is used to obtain instructions from the driving module 140 and provide rotational movement for the second simulated eyeball module 200 to simulate the actual movement of the human left eye, such as gaze, eye saccade and tracking. The second rotation module 220 is composed of a second rotation drive component 221 and a second rotation transmission component 222. Preferably, the second rotation module 220 also has an error compensation device, such as a photoelectric encoder, a Hall sensor, or a magnetoresistive sensor, or an inertial sensor, etc., for closed-loop control of the VR / AR eye tracking test device. Further preferably, the second rotation module 220 has a matte black surface coating to facilitate the VR / AR device eye tracking device to identify the second simulated eyeball module 200 for eye tracking testing.

[0074] The second rotation drive component 221 can be a DC motor, a stepper motor, or a servo motor. The second rotation drive component 221 is connected to the second rotation transmission component 222 by a key connection or a coupling connection. The second rotation drive component 221 is connected to the second rotation module base 132 by a threaded connection, a welding connection, or an integral connection with the second rotation module base 132.

[0075] The second rotating transmission component 222 can be a gear transmission, a worm gear mechanism, or a shaft. The second rotating transmission component 222 is connected to the second pitch module bracket 213, and the connection method can be flange connection, welding, or integration with the second rotating transmission component 222, so that the second simulated eyeball module 200 rotates around its rotation axis.

[0076] Figure 2 This is a structural schematic diagram of the simulated eyeball module of Example 1 and Example 2 of the eye tracking test device and test method for VR / AR equipment of the present invention, comprising a simulated cornea 101, a simulated iris 102, a simulated sclera 103, a human eye lens 104, an image sensor 105, and a posture sensor 106.

[0077] The simulated cornea 101 is used to simulate the human cornea participating in refraction, and to penetrate light into the simulated sclera 103, and at the same time reflect the light generated by the light source of the eye tracking device in the VR / AR device to generate a Purkin spot, so that the eye tracking device in the VR / AR device can identify the spot to obtain eye movement data. The simulated cornea 101 can be glass, transparent resin, or a convex-concave lens made of silicone gel. The simulated cornea 101 is located outside the front hemisphere of the simulated sclera 103 and is connected to the simulated sclera 103. The connection method can be glue connection, clip connection, or integration with the simulated sclera 103. Preferably, the manufacturing material of the simulated cornea 101 has a refractive index greater than or equal to 1.1, and a transmittance higher than 70%. The front surface curvature radius of the simulated cornea 101 is less than 9 mm, the rear surface curvature radius range is less than or equal to 8 mm, the front surface refractive power is less than or equal to +50D, the rear surface refractive power range is greater than or equal to -7D, and the surface transverse diameter and surface longitudinal diameter of the simulated cornea 101 are less than or equal to 13 mm. Further preferably, the refractive index range of the material for making the simulated cornea 101 is 1.37 to 1.38, the transmittance range is 97% to 98%, the front surface curvature radius range is 7.7mm to 7.9mm, the back surface curvature radius range is 6.7mm to 6.9mm, the front surface refractive power range is +48D to +49D, the back surface refractive power range is -5D to -6D, the total refractive power range is +42D to +44D, the surface transverse diameter range is 11mm to 13mm, and the surface longitudinal diameter range is 10.5mm to 11.5mm, so as to match the corneal shape and refractive power of a normal human eye, thereby more accurately simulating the human cornea.

[0078] The simulated iris 102 is used to simulate the color of the iris of the human eye and the changes in the pupil diameter. The simulated iris 102 has two functions. On the one hand, the simulated iris 102 receives the instructions of the drive control module 150 in a timely manner, adjusts the pupil diameter of the first simulated eyeball module 100 or the second simulated eyeball module 200 according to the test task type and scene changes, and simulates the human eye to adjust the light intensity entering the eye according to the intensity of the external light; on the other hand, the simulated iris 102 is used for the eye tracking device in the VR / AR device to identify the pupil, so that the eye tracking device in the VR / AR device can obtain the pupil center coordinates and pupil diameter. The simulated iris 102 is an aperture with an adjustment structure with a non-white surface, which can be a variable aperture with a mechanical adjustment structure, a variable aperture with an electric adjustment mechanism, or a transmissive display, wherein the transmissive display can be a transmissive LCD display, a transmissive LED display, or a transmissive OLED display. The simulated iris 102 is located between the simulated cornea 101 and the human eye lens 104, and is connected to the human eye lens 104, and the connection method can be connected by thin transparent glue or cage plate and cage rod. The simulated iris 102 is connected to the drive control module 150, and the connection method can be a wired connection such as USB, serial port, parallel port, etc., or a wireless connection such as Bluetooth, local area network, LoRa, etc. Preferably, the pupil variation range simulated by the simulated iris 102 is 2mm to 7mm, so as to more accurately simulate the effect of the human eye iris, and facilitate the eye tracking device in the VR / AR device to detect and identify the edge of the pupil, iris and sclera of the first simulated eyeball module 100 or the second simulated eyeball module 200 to obtain information such as pupil center coordinates, eyeball optical axis vector and visual axis vector.

[0079] The simulated sclera 103 is used to simulate the protection of the eyeball by the sclera of the human eye, and accommodates the simulated iris 102, the human eye lens 104 and the image sensor 105 so that these components are not affected by the external environment. At the same time, it is used together with the simulated iris 102 for the eye tracking device in the VR / AR device to identify the pupil edge to obtain the pupil center coordinates and pupil diameter. The simulated sclera 103 can be a white object made of metal material or non-metallic material. The rotation center of the first simulated eyeball module 100 or the second simulated eyeball module 200 is used as the origin O, and the pitch axis of the first simulated eyeball module 100 or the second simulated eyeball module 200 is used as the x-axis to establish a spatial coordinate system according to the right-hand spiral. With the z-axis as the reference, the simulated sclera 103 can be divided into the front hemisphere in the +z direction and the rear hemisphere in the -z direction. The simulated sclera 103 is connected to the simulated cornea 101, and the connection method can be glue connection, clip connection, or integrated with the simulated sclera 103. The simulated sclera 103 is connected to the first pitch transmission component 112 or the second pitch transmission component 212, and the connection method can be a coupling connection or a key connection. The simulated sclera 103 is connected to the image sensor 105, and the connection method can be glue connection, snap connection, or integrated with the simulated sclera 103. The simulated sclera 103 is connected to the posture sensor 106, and the connection method can be snap connection or glue connection. Preferably, the simulated sclera 103 is an opaque white spherical crown shell with an outer surface diameter ranging from 23mm to 26mm and a spherical crown bottom radius ranging from 10mm to 13mm, so as to more accurately match the surface transverse diameter and surface longitudinal diameter of the simulated cornea 101, so as to facilitate the eye tracking device in the VR / AR device to detect and identify the edge of the pupil, iris and sclera of the first simulated eyeball module 100 or the second simulated eyeball module 200 to obtain information such as pupil center coordinates, eyeball optical axis vector and visual axis vector. Further preferably, the simulated sclera 103 can also be a drum-shaped body with an outer surface diameter ranging from 23mm to 26mm and a shell thickness ranging from 2mm to 3mm, the radius of the bottom of the front hemisphere cap ranging from 10mm to 13mm, and the radius of the rear hemisphere cap ranging from 11mm to 12mm, so as to facilitate the fixation of the image sensor 105 to the simulated sclera 103 and the connection of the image sensor 105 to the main control module 160.

[0080] The human eye lens 104, the simulated cornea 101, the simulated iris 102 and the image sensor 105 form a coaxial optical system, and the optical axis of the system is the optical axis of the human eye lens 104. The human eye lens 104 has two functions. On the one hand, the human eye lens 104 is located in front of the image sensor 105, and is used to simulate the imaging process of the lens in the human eye onto the retina; on the other hand, the human eye lens 104 has a module for adjusting the focal length. By synchronously receiving the instructions of the driving control module 150, the focal length of the first simulated eyeball module 100 or the second simulated eyeball module 200 can be adjusted according to the test task type and scene changes, so that the image sensor 105 can clearly image the object located at the spatial gaze point coordinates at the current moment. The human eye lens 104 can be glass, transparent plastic, or a lens group made of fluorite, wherein the lens group can be a lens-reflector combination structure composed of multiple single lenses or reflectors, such as an in-line front aperture eyepiece lens group, a super-heart in-line front aperture lens group, or a turning front aperture lens-reflector group, etc., or a single lens structure. The human eye lens 104 includes at least one positive focal length lens. The module for adjusting the focus in the human eye lens 104 can be a liquid crystal liquid lens, an electrowetting liquid lens, a liquid-filled liquid lens, or a lens group with a mechanical zoom structure. The focus adjustment of the human eye lens 104 can be manual or electric. The human eye lens 104 is connected to the simulated iris 102, and the connection method can be connected by using a thin transparent glue or a cage plate and cage rod. The human eye lens 104 is connected to the image sensor 105, and the connection method can be connected by using a cage plate and cage rod, or it can be snap-connected, or it can be integrated with the image sensor 105. The human eye lens 104 is connected to the driving control module 150, and the connection mode can be a wired connection such as USB, serial port, parallel port, etc., or a wireless connection such as Bluetooth, local area network, LoRa, etc. Preferably, the field of view angle θ of the human eye lens 104 ranges from ±30° to ±90°, the MTF cutoff frequency is greater than 40lp / mm, the Strehl ratio (SR) is greater than 0.15, the entrance pupil diameter ranges from 2mm to 7mm, the field of view ranges from ±60° to ±80°, the exit pupil distance ranges from 10mm to 15mm, the front focal length ranges from 10mm to 15mm, and the back focal length ranges from 20mm to 30mm, so as to more realistically simulate the best viewing angle of the human eye.

[0081] The image sensor 105 is used to receive the optical signal of the human eye lens 104 and convert it into an image, and synchronously transmit the images collected when the first simulated eyeball module 100 and the second simulated eyeball module 200 move to the main control module 160 for display, simulating the collection, processing and transmission of optical signals by the retina of the human eye. The image sensor 105 is located in the rear hemisphere of the simulated sclera 103 and in the pupil box of the human eye lens. The image sensor 105 can be planar or curved. The pixels of the image sensor 105 can be evenly distributed or unevenly distributed. The image sensor 105 is connected to the human eye lens 104, and the connection method can be connected by cage plates and cage rods, or it can be snap-fitted, or it can be integrated with the human eye lens 104. The image sensor 105 is connected to the simulated sclera 103, and the connection method can be glue connection, or it can be snap-fitted, or it can be integrated with the simulated sclera 103. The image sensor 105 is connected to the main control module 160, and the connection method can be connected through video interfaces such as VGA, HDMI, DVI, DisplayPort, USB, serial port, and general I / O. Preferably, the frame rate of the image sensor 105 can be changed between 24 frames per second and 240 frames per second, and its pixel density can decrease from the center to the periphery, the pixel density range of the central visual area is 300ppi to 450ppi, the peripheral visual area range is 250ppi to 300ppi, and the radius range of the central visual area is 1.4mm to 1.6mm, so as to simulate the equivalent pixel density and photosensitivity effect of the fovea area and other areas of the human eye retina. Further preferably, the maximum circumscribed circle radius range of the image sensor 105 should be less than 13mm to simulate the horizontal and vertical diameters of the human eye retina in the eyeball. Further preferably, the shutter speed of the image sensor 105 should be less than or equal to 1 / 43200s to match the limiting resolution angle of the human eye.

[0082] The posture sensor 106 can be an inertial sensor, an angle sensor, or a gyroscope. The posture sensor 106 has three functions. On the one hand, the posture sensor 106 is used to synchronously measure the motion data of the first simulated eyeball module 100 or the second simulated eyeball module 200, such as rotation angle, pitch angle, rotation angular velocity, pitch angular velocity, rotation angular acceleration, and pitch angular acceleration; on the other hand, the posture sensor 106 synchronously transmits the motion data to the main control module 160, and performs angle compensation for the motion synchronization of the VR / AR eye tracking test device according to the test task; on the other hand, when the test task is determined, the posture sensor 106 can obtain the angle data of the first simulated eyeball module 100 and the second simulated eyeball module 200 after the previous test task is completed, and transmit the angle data to the main control module 160 for device reset. The posture sensor 106 is located in the rear hemisphere of the simulated sclera and behind the image sensor. The posture sensor 106 is connected to the simulated sclera 103, and the connection method can be snap-on or glue-connected. The attitude sensor 106 is connected to the main control module 160, and the connection mode can be a wired connection such as USB, serial port, parallel port, etc., or a wireless connection such as Bluetooth, local area network, LoRa, etc. Preferably, the attitude sensor 106 can use an inertial measurement unit, whose maximum zero point offset range is less than or equal to ±0.1%, zero bias stability range is less than or equal to 0.01° / h, sensitivity range is less than or equal to 10mV / ° / s, and proportional factor error is less than or equal to 0.5%, so as to accurately control the rotation angle and pitch angle of the first simulated eyeball module 100 or the second simulated eyeball module 200.

[0083] In this embodiment, the VR / AR device eye tracking test device can perform vertical pitch movement, horizontal rotation movement, pupil distance change, and image acquisition and processing on two simulated eyeball modules, and can accurately simulate various actual movements of the human eye with different pupil distances, such as gaze, eye saccade, and tracking, and process and display the images obtained during movement according to the simulated human eye characteristics.

[0084] Example 2

[0085] An embodiment of the present invention is an eye tracking test device and a test method for VR / AR equipment, such as Figure 3As shown, the eye tracking test device for VR / AR equipment includes a first simulated eyeball module 100, a first pitch module 110, a first pitch driving component 111, a first pitch transmission component 112, a first pitch module bracket 113, a first rotation module 120, a first rotation driving component 121, a first rotation transmission component 122, a pupil distance control module 130, a first rotation module base 131, a second rotation module base 132, a drive module 140, a drive control module 150, a main control module 160, a second simulated eyeball module 200, a second pitch module 210, a second pitch driving component 211, a second pitch transmission component 212, a second pitch module bracket 213, a second rotation module 220, a second rotation driving component 221, and a second rotation transmission component 222.

[0086] The first simulated eyeball module 100 and the second simulated eyeball module 200 are respectively used to simulate the actual movement of the right eye and the left eye of a person, and synchronously transmit the image data and the movement data obtained by the movement to the main control module 160. The first simulated eyeball module 100 and the second simulated eyeball module 200 are both composed of a simulated cornea 101, a simulated iris 102, a simulated sclera 103, a human eye lens 104, an image sensor 105 and a gesture sensor 106.

[0087] The first pitch module 110 is used to obtain the instruction of the driving module 140 and provide pitch motion for the first simulated eyeball module 100. The first pitch module 110 is composed of a first pitch driving component 111 and a first pitch transmission component 112.

[0088] The first pitch driving component 111 is connected to the first pitch module bracket 113. The first pitch driving component 111 is connected to the first pitch transmission component 112, and the connection mode can be key connection or coupling connection.

[0089] The first pitch transmission component 112 may be a synchronous pulley set, a chain sprocket set, or a V-belt pulley set. The first pitch transmission component 112 is connected to the simulated sclera 103 in the first simulated eyeball module 100, and the connection method may be a coupling connection or a key connection. Preferably, the distance between the input wheel center axis and the output wheel center axis of the first pitch transmission component 112 is 16 cm to 30 cm.

[0090] The first pitch module bracket 113 is connected to the first pitch driving component 111 and the first rotation transmission component 122, so that the first simulated eyeball module 100 rotates around the rotation axis and the pitch axis.

[0091] The first rotation module 120 is used to obtain the instruction of the driving module 140 and provide the first simulated eyeball module 100 with a rotational motion. The first rotation module 120 is composed of a first rotation driving component 121 and a first rotation transmission component 122 .

[0092] The first rotation driving component 121 is connected to the first rotation module base 131. The first rotation driving component 121 is connected to the first rotation transmission component 122, and the connection mode can be a key connection or a coupling connection.

[0093] The first rotating transmission component 122 may be a synchronous pulley set, a chain sprocket set, or a V-belt pulley set. The first rotating transmission component 122 is connected to the first pitch module bracket 113, and the connection method may be a flange connection, welding, or integration with the first rotating transmission component 122, so that the first simulated eyeball module 100 rotates around its rotation axis. Preferably, the distance between the input wheel center axis and the output wheel center axis of the first rotating transmission component 122 is 16 cm to 30 cm.

[0094] The pupil distance control module 130 is used to provide different pupil center distances for the first simulated eyeball module 100 and the second simulated eyeball module 200. The pupil distance control module 130 is composed of a first rotating module base 131, a second rotating module base 132 and a pupil distance controller 133. The first rotating module base 131 is used to fix the first rotating drive component 121 on the pupil distance control module 130. The first rotating module base 131 is connected to the first rotating drive component 121. The second rotating module base 132 is used to fix the second rotating drive component 221 on the pupil distance control module 130. The second rotating module base 132 is connected to the second rotating drive component 221. The pupil distance controller 133 is used to receive the command program of the drive control module 150 and adjust the pupil center distances of the first simulated eyeball module 100 and the second simulated eyeball module 200 according to the command. The pupil distance controller 133 is connected to the first rotating module base 131, the second rotating module base 132 and the drive module 140.

[0095] The driving module 140 is a host computer of the first pitch driving component 111, the first rotation driving component 121, the pupil distance controller 133, the second pitch driving component 211, the second rotation driving component 221 and a slave computer of the driving control module 150, and is used to obtain command programs or data from the driving control module 150 and control corresponding modules or components to execute corresponding instructions. The driving module 140 is connected to the first pitch driving component 111, the first rotation driving component 121, the pupil distance controller 133, the second pitch driving component 211, the second rotation driving component 221 and the driving control module 150 respectively.

[0096] The drive control module 150 is the upper computer of the drive module 140 and the lower computer of the main control module 160, and is used to obtain the command program or data transmitted from the main control module 160, and control the drive module 140 to drive the pitch drive component, the rotation module 120, and the pupil distance control module 130 to execute the corresponding command. The drive control module 150 is connected to the simulated sclera 102, the human eye lens 104, the drive module 140, and the main control module 160 respectively.

[0097] The main control module 160 is used to control the driving control module 150, obtain and display the image transmitted by the image sensor 105 in the first simulated eyeball module 100 and the second simulated eyeball module 200, and perform angle compensation on the first simulated eyeball module 100 and the second simulated eyeball module 200 according to the motion data transmitted by the posture sensor 106 in the first simulated eyeball module 100 and the second simulated eyeball module 200. The main control module 160 is connected to the driving control module 150, the image sensor 105 and the posture sensor 106 in the first simulated eyeball module 100 and the second simulated eyeball module 200, respectively.

[0098] The second pitch module 210 is used to obtain the instruction of the driving module 140 and provide pitch motion for the second simulated eyeball module 200. The second pitch module 210 is composed of a second pitch driving component 211 and a second pitch transmission component 212.

[0099] The second pitch driving component 211 is connected to the second pitch module bracket 213. The second pitch driving component 211 is connected to the second pitch transmission component 212, and the connection mode can be key connection or coupling connection.

[0100] The second pitch transmission component 212 may be a synchronous pulley set, a chain sprocket set, or a V-belt pulley set. The second pitch transmission component 212 is connected to the simulated sclera 103 in the second simulated eyeball module 200, and the connection method may be a coupling connection or a key connection. Preferably, the distance between the input wheel center axis and the output wheel center axis of the second pitch transmission component 212 is 16 cm to 30 cm.

[0101] The second pitch module bracket 213 is connected to the second pitch driving component 211 and the second rotation transmission component 222, so that the second simulated eyeball module 200 rotates around the rotation axis and the pitch axis.

[0102] The second rotation module 220 is used to obtain the instruction of the driving module 140 and provide rotational motion for the second simulated eyeball module 200. The second rotation module 220 is composed of a second rotation driving component 221 and a second rotation transmission component 222.

[0103] The second rotation driving component 221 is connected to the second rotation module base 132. The second rotation driving component 221 is connected to the second rotation transmission component 222, and the connection mode can be a key connection or a coupling connection.

[0104] The second rotating transmission component 222 can be a synchronous pulley set, a chain sprocket set, or a V-belt pulley set. The second rotating transmission component 222 is connected to the second pitch module bracket 213, and the connection method can be flange connection, welding, or integration with the second pitch module bracket 213, so that the second simulated eyeball module 200 rotates around its rotation axis. Preferably, the distance between the input wheel center axis and the output wheel center axis of the second rotating transmission component 222 is 16 cm to 30 cm.

[0105] In this embodiment, the VR / AR device eye tracking test device can perform vertical pitch movement, horizontal rotation movement, pupil distance change, and image acquisition and processing on two simulated eyeball modules, and can accurately simulate various actual movements of human eyes with different pupil distances, such as gaze, eye saccade, and tracking, and process and display the images collected during movement according to the simulated human eye characteristics. At the same time, this embodiment reduces the size of the device through the pitch transmission component and the rotation transmission component, making this embodiment easier to operate and use in the eye tracking test of small-volume VR / AR devices, and providing stronger portability for the VR / AR device eye tracking test.

[0106] Embodiments 1 and 2 of the present invention provide an eye tracking test method for VR / AR devices, such as Figure 4 The method comprises:

[0107] Step 1: Place the VR / AR device to be tested in front of the testing device so that the first simulated eyeball module and the second simulated eyeball module are located in the pupil box of the VR / AR device to be tested.

[0108] Step 2: Set the VR / AR device eye tracking test task type and test trajectory in the main control module. According to the eye tracking test task type, set the image sensor frame rate, initial rotation position, initial pitch position, maximum rotation angle, maximum pitch angle, maximum rotation angular velocity, maximum pitch angular velocity, pupil center distance p, gaze time and other test parameters.

[0109] It should be noted that the test trajectory can be trajectory data from a real-person VR / AR device eye tracking test, or it can be trajectory data generated by a computer and contain one or more human eye movements such as gaze, saccade and tracking, so that the test device can truly simulate the eye movement of a real person when performing a VR / AR device eye tracking test.

[0110] Step 3: Reset the test device, and the main control module sends the motion data of each module to the drive control module, so that the drive module drives the corresponding module in the test device to move.

[0111] It should be noted that while performing motion control, the drive control module can control the human eye lenses in the first simulated eyeball module and the second simulated eyeball module to adjust the focal length according to the scene changes and the current spatial gaze point position to clearly display the objects that need to be paid attention to, and control the simulated irises in the first simulated eyeball module and the second simulated eyeball module to adjust the pupil size to simulate the human eye adjusting the intensity of the incoming light according to the intensity of the external light.

[0112] Step 4: The main control module synchronously displays the images captured by the first simulated eyeball module and the second simulated eyeball module at the current moment, the spatial gaze point coordinate position G (x, y, z) at the current moment, and the gaze point coordinates given by the eye tracking device in the VR / AR device.

[0113] It should be noted that the timing synchronization of displaying the captured image, displaying the spatial gaze point coordinate information at the current moment, and displaying the gaze point coordinates in the eye tracking device of the VR / AR device can be achieved by synchronization at the hardware level, such as using the synchronization signal or clock signal generated by the drive control module for synchronization, or by synchronization at the software level, such as using a clock synchronization program in the main control module, using network time synchronization, etc.

[0114] Step 5: Repeat steps 2 to 4 until all test tasks are completed. VR / AR eye tracking test results are given based on the obtained coordinate position comparison information of all gaze points, such as viewing angle accuracy, delay time, gaze point mean square error, etc.

[0115] The present invention also provides a control method for an eye tracking test device for VR / AR equipment, which can be used to control the first simulated eyeball module and the second simulated eyeball module of the eye tracking test device to accurately simulate human eye movement, such as Figure 5 The method comprises:

[0116] Step 1: The posture sensor obtains the angle data of the first simulated eyeball module and the second simulated eyeball module after the test task and test trajectory are set. The main control module compares the current angle data with the set initial angle data, and sends instructions to the drive control module to reset the first pitch module, the first rotation module, the second pitch module and the second rotation module.

[0117] Step 2: The main control module sends instructions to the drive control module according to the pupil center distance set in the test task, controls the pupil distance control module to adjust the pupil center distance of the two simulated eyeball modules, and splits the trajectory into motion data of the first pitch module, the first rotation module, the second pitch module and the second rotation module in time series according to the set VR / AR device eye tracking test trajectory.

[0118] It should be noted that the time series can be derived from trajectory data of eye tracking tests of real-life VR / AR devices, or from trajectory data generated by a computer that contains one or more human eye movements such as gaze, saccades, and following. It can also be derived from synchronization signals at the hardware level, such as synchronization signals or clock signals generated by a drive control module, or from synchronization programs at the software level, such as using a clock synchronization program in a main control module, using network time synchronization, etc.

[0119] Step 3: The main control module sends the motion data to the drive control module in a timely manner, so that the corresponding module drives the first simulated eyeball module and the second simulated eyeball module to perform pitch motion or rotation motion.

[0120] In this step, the spatial three-dimensional gaze point coordinates at a certain moment in the test trajectory can be converted into the rotation angle and pitch angle of the first simulated eyeball module and the second simulated eyeball module by establishing a spatial rectangular coordinate system. Figure 6 As shown, let the pupil center of the first simulated eyeball module be A, and the pupil center of the second simulated eyeball module be B, then AB=p, where p is the set pupil center distance. Let the initial rotation angle of the first simulated eyeball module be α 0 , the initial rotation angle of the second simulated eyeball module is β 0 , the initial pitch angle of the simulated eyeball module is γ 0. Taking the midpoint of the line AB as the origin O of the spatial rectangular coordinate system, and the line connecting the two points AB as the x-axis, the spatial rectangular coordinate system is established according to the right-hand spiral, and the coordinates of the two pupil centers A(0.5p,0,0) and B(-0.5p,0,0) can be obtained. According to the current target gaze point G(x,y,z) and the set pupil distance p, the required rotation angle of the first simulated eyeball module can be calculated through the trigonometric function relationship. α=arctan((x-0.5p) / z)-α 0 , the rotation angle of the second simulated eyeball module β = arctan ((x + 0.5p) / z) - β 0 And the pitch angle of the two simulated eyeball modules γ = arctan (y / z) - γ 0 .

[0121] It should be noted that the timing transmission of the motion data is also synchronized with the timing display of the captured image, the actual space gaze point coordinate information and the gaze point coordinates in the VR / AR device eye tracking test device control method by the main control device. For example, when the main control module acquires and displays the captured image, the actual space gaze point coordinate information and the gaze point coordinates in the VR / AR device eye tracking device at the i-th moment, the motion data at the i+1th moment is sent to the drive control module at the same time.

[0122] Step 4: The posture sensor detects the pitch angle and rotation angle of the current first simulated eyeball module and the second simulated eyeball module. The main control module compares the current rotation angle and pitch angle with the required rotation angle and pitch angle, so that the drive control module controls the corresponding module to perform angle compensation.

[0123] Step 5: Repeat steps 3 and 4 until the test track is completed.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Under the concept of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present invention as described above. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An eye tracking test device for VR / AR equipment, characterized in that: include: The simulated eyeball module is used to simulate the movement of the human eye and is equipped with a posture sensor for detecting the pitch angle and rotation angle of the simulated eyeball module in real time; A pitch module, connected to the simulated eye module, and used to drive the simulated eye module to achieve pitch movement; A rotation module, connected to the simulated eyeball module, and used to drive the simulated eyeball module to achieve rotational motion; A pupil distance control module, used for adjusting the simulated eyeball module to simulate the pupil distance of different users; A driving module, connected to the pitch module, the rotation module and the pupil distance control module respectively, for providing power to perform corresponding motion control; A drive control module, connected to the attitude sensor and the drive module respectively, for receiving pitch angle and rotation angle information of the simulated eyeball module detected by the attitude sensor, and controlling the drive module to perform error compensation based on the information; The main control module is respectively connected to the drive control module, the posture sensor and the VR / AR device eye tracking system, and is used to set the test task type and the test trajectory, receive the gaze point given by the VR / AR device eye tracking system, and compare the gaze point currently obtained by the simulated eye module through the VR / AR device eye tracking system with the set gaze point, so as to achieve accuracy analysis of the VR / AR device eye tracking system.

2. The eye tracking test device for VR / AR equipment according to claim 1, characterized in that: The simulated eyeball module comprises: a first simulated eyeball module (100) and a second simulated eyeball module (200), wherein the first simulated eyeball module (100) and the second simulated eyeball module (200) are both composed of a simulated cornea (100), a simulated iris (102), a simulated sclera (103), a human eye lens (104), an image sensor (105) and a posture sensor (106), wherein the simulated cornea (100), the simulated iris (102), the human eye lens (104) and the image sensor (105) form a coaxial optical system with the optical axis of the human eye lens as the axis; the first simulated eyeball module (100) and the second simulated eyeball module (200) are capable of reflecting light generated by a light source of an eye tracking device in a VR / AR device to generate a Purkinje light spot for the eye tracking device to identify the light spot and obtain eye movement data, and The instructions of the drive control module (150) are received, and the pupil diameter and focal length are adjusted according to the test task type and scene changes, so as to simulate the human eye adjusting the light intensity entering the eye according to the strength of the external light, and facilitate the pupil recognition of the eye tracking device in the VR / AR device; the image sensor (104) clearly images the object located at the spatial gaze point coordinates at the current moment, and synchronously transmits the image collected during the movement to the main control module (160) for display, so as to simulate the collection, processing and transmission of light signals by the retina of the human eye; the posture sensor (106) synchronously measures the movement data of the first simulated eyeball module (100) or the second simulated eyeball module (200) and synchronously transmits it to the main control module (160) to synchronously perform angle compensation on the movement of the test device, so as to accurately simulate the human binocular viewing environment and different types of actual movements; The pitch module comprises a first pitch module and a second pitch module; wherein the first pitch module (110) comprises a first pitch driving component (111), a first rotating transmission component (112) and a first pitch module bracket (113) connecting the first pitch driving component (111) and the first rotating transmission component (112); the second pitch module (210) comprises a second pitch driving component (211), a second pitch transmission component (212) and a second pitch module bracket (213) connecting the second pitch driving component (211) and the second rotating transmission component (212); the first pitch module (110) is used to obtain instructions from the driving module, drive the first simulated eyeball module to perform pitch movement around the pitch axis, and simulate the actual movement of the right eye of a person to gaze, saccade and follow; the second pitch module (210) is used to obtain instructions from the driving module, drive the second simulated eyeball module to perform pitch movement around the pitch axis, and simulate the actual movement of the left eye to gaze, saccade and follow; The rotation module comprises a first rotation module (120) and a second rotation module (220), wherein the first rotation module (120) is composed of a first rotation driving component (121) and a first rotation transmission component (122), and is used to obtain instructions from the driving module, drive the first simulated eyeball module (100) to perform rotational motion around a rotation axis, and simulate the actual motion of a human right eye; the second rotation module (220) is composed of a second rotation driving component (221) and a second rotation transmission component (222), and is used to obtain instructions from the driving module, drive the second simulated eyeball module (210) to perform rotational motion around a rotation axis, and simulate the actual motion of a human left eye; The pupil distance control module (130) comprises a first rotating module base (131), a second rotating module base (132) and a pupil distance controller (133), wherein the first rotating module base (131) is connected to the first rotating drive component (121), the second rotating module base (132) is connected to the second rotating drive component (221), and the pupil distance controller (133) is respectively connected to the first rotating module base (131), the second rotating module base (132) and the driving module (140), and is used for adjusting the pupil center distance of the first simulated eyeball module (100) and the second simulated eyeball module (200) according to the command or data of the driving module, so as to simulate the pupil distance of different people, and the variation range is 40 mm to 80 mm.

3. The eye tracking test device for VR / AR equipment according to claim 2, characterized in that: The first pitch driving component (111), the first rotation driving component (121), the second pitch driving component (211) and the second rotation driving component (221) are DC motors, stepping motors or servo motors; the first pitch transmission component (112), the first rotation transmission component (122), the second pitch transmission component (212) and the second rotation transmission component (222) are gear transmissions, worm gear mechanisms, shafts, synchronous pulley sets, chain sprocket sets or V-belt pulley sets.

4. The eye tracking test device for VR / AR equipment according to claim 2, characterized in that: The image sensor (105) is located in the rear hemisphere of the simulated sclera (102) and in the pupil box of the human eye lens (104); the simulated iris (102) is a stop with a non-white surface and an adjustment structure; the human eye lens (104) has a focal length adjustment module and at least one positive focal power lens.

5. The eye tracking test device for VR / AR equipment according to claim 2, characterized in that: The simulated cornea (101) is a convex-concave lens made of glass, transparent resin, or silicone gel.

6. The eye tracking test device for VR / AR equipment according to claim 2, characterized in that: The simulated iris (102) is a variable iris with a mechanical adjustment structure, or a variable iris with an electric adjustment mechanism, or a transmissive display, or a transmissive LCD display, or a transmissive LED display, or a transmissive OLED display.

7. The eye tracking test device for VR / AR equipment according to claim 1, characterized in that: The driving module (140) is used to obtain the command program or data from the driving control module (150) and control the corresponding module or component to execute the corresponding instruction; the driving control module (150) is used to obtain the command program or data transmitted by the main control module (160), synchronously control the driving module (140) to drive the corresponding module or component to execute the instruction, and synchronously control the human eye lens (104) and the simulated sclera (103) in the first simulated eyeball module (100) and the second simulated eyeball module (200) to adjust the focal length and pupil size according to the test task and scene changes; and synchronize the timing information with the main control module (160) and synchronously send the spatial gaze point coordinate information at the current moment to the main control module (160); the main control module (160) is used to decompose the test trajectory into motion data of each module. The method comprises: collecting data from the image sensors (105) in the first simulated eyeball module (100) and the second simulated eyeball module (200), and synchronously sending instructions to the drive control module (150) to control each module; and editing and processing the images input by the image sensors (105) in the first simulated eyeball module (100) and the second simulated eyeball module (200) according to the test task to simulate the sensitivity of different human eyes to information such as color and brightness in the environment; and synchronously acquiring and displaying the images transmitted by the image sensors (105) in the first simulated eyeball module (100) and the second simulated eyeball module (200), the spatial gaze point coordinates at the current moment, and the spatial gaze point coordinates given by the eye tracking device of the VR / AR device; and synchronously controlling the drive control module (150) to perform angle compensation on the first simulated eyeball module (100) and the second simulated eyeball module (200) according to the motion data transmitted by the posture sensor (106).

8. A test method for eye tracking of VR / AR devices, characterized in that: The following steps are involved: S1. Place the VR / AR device to be tested in front of the test device according to any one of claims 1 to 7, so that the simulated eyeball module is located in the pupil box of the VR / AR device to be tested; S2. Set the VR / AR device eye tracking test task type and test trajectory through the main control module; S3 drives the pitch module and the rotation module to make the simulated eyeball module perform pitch and rotation movements according to the set test trajectory; S4. Use the attitude sensor to detect the pitch angle and rotation angle of the simulated eyeball module in real time, and perform error compensation through the drive control module; S5. The main control module receives the gaze point information given by the VR / AR device eye tracking system based on the simulated eyeball module movement; S6. Compare the gaze point of the current simulated eyeball module with the gaze point given by the eye tracking system of the VR / AR device, and analyze the accuracy of the eye tracking system of the VR / AR device.

9. The test method for eye tracking of VR / AR devices according to claim 8, characterized in that: The method further includes the steps of: adjusting the pupil distance of the simulated eyeball module through the pupil distance control module according to the test requirements to simulate the eye characteristics of different users for testing; The image acquisition module is used to collect image information of the simulated eye module in the VR / AR environment in real time to further analyze the performance of the VR / AR device eye tracking system.

10. The test method for eye tracking of VR / AR devices according to claim 8, characterized in that: The step S4. specifically comprises: S4.1 The posture sensor (106) obtains the angle data of the first simulated eyeball module (100) and the second simulated eyeball module (200) after the test task and the test track are set, and the main control module (160) compares the current angle data with the set initial angle data, and controls the drive control module (150) to reset the corresponding module; S4.2 The main control module (160) sends instructions to the driving control module (150) according to the pupil center distance set in the test task, controls the pupil distance control module (130) to adjust the pupil center distance of the two simulated eyeball modules, and divides the trajectory into motion data of each module in time series according to the set VR / AR device eye tracking test trajectory. S4.3 the main control module (160) sends the motion data to the drive control module (150) in a timely manner, so that the corresponding module drives the first simulated eyeball module (100) and the second simulated eyeball module (200) to perform pitch motion or rotation motion; S4.4 The posture sensor (106) detects the current pitch angle and rotation angle of the first simulated eyeball module (100) and the second simulated eyeball module (200), and the main control module (160) compares the current rotation angle and pitch angle with the required rotation angle and pitch angle, so that the drive control module (150) controls the corresponding module to perform angle compensation; S4.5 is repeated in a loop until the test track is completed.

Images