Delay time determination method and device, electronic equipment and storage medium

Abstract

The invention provides a delay time determination method and apparatus, an electronic device and a storage medium, the method is used for determining the delay time of eye movement tracking in a to-be-tested XR device, and the method comprises the steps of receiving output information of a sensor, determining a first moment when an artificial eye moves to a preset position according to the output information, the output information is used for representing actual motion information of the artificial eye; the time when the synchronous signal is received is determined as a second time, the synchronous signal is a signal output when the to-be-tested XR device determines that the artificial eye moves to a preset position through the eye movement tracking module, first delay time is determined according to the first time and the second time, and the time point when the artificial eye moves to the preset position is captured through the sensor; and capturing a time point of a synchronization signal generated when the eye movement tracking module determines that the artificial eye moves to a preset position, thereby accurately determining delay from real occurrence of the eye movement event to detection of the eye movement event by the to-be-tested XR equipment through an algorithm, and determining a performance index of the to-be-tested XR equipment.

Description

Technical Field

[0001] This invention relates to the field of signal processing technology, and in particular to a method, apparatus, electronic device, and storage medium for determining delay time. Background Technology

[0002] Eye-tracking technology is a technique used to measure and record eye movements. It can track information such as the user's gaze point, eye movement trajectory, and pupil changes, and is a key technology in the field of human-computer interaction in XR (Extended Reality) devices.

[0003] The real-time nature of eye tracking determines the human-computer interaction experience. If the delay is too long, it will lead to adverse consequences such as the misalignment between virtual and reality, difficulty in human-computer interaction, and dizziness in users.

[0004] Therefore, it is necessary to measure the delay time of eye tracking in XR devices in order to clarify the performance indicators of XR devices. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for determining delay time, used to accurately measure the delay time of eye tracking in XR devices.

[0006] In a first aspect, the present invention provides a method for determining delay time, the method being used to measure the delay time of eye tracking in an XR device under test, wherein the eye tracking module in the XR device under test is used to detect eye movement events of an artificial eye; the method includes:

[0007] The system receives output information from the sensor and determines the first moment when the artificial eye moves to a preset position based on the output information; the output information is used to represent the actual motion information of the artificial eye.

[0008] The moment when the synchronization signal is received is defined as the second moment; the synchronization signal is the signal output by the XR device under test when the artificial eye moves to the preset position, as determined by the eye-tracking module.

[0009] The first delay time is determined based on the first time point and the second time point;

[0010] Wherein, the first delay time is the delay from the occurrence of the eye movement event to the detection of the eye movement event by the XR device under test.

[0011] Optionally, the sensor is an inertial measurement unit, and the output information is acceleration and angular velocity at different times; determining the first moment when the artificial eye moves to the preset position based on the output information includes:

[0012] Based on the acceleration and angular velocity at different times, the first moment when the artificial eye moves to the preset position is determined.

[0013] Optionally, the sensor is a motor encoder, and the output information is A-phase and B-phase pulses at different times; determining the first moment when the artificial eye moves to the preset position based on the output information includes:

[0014] The first moment when the artificial eye moves to the preset position is determined based on the number or phase relationship of pulses at different times.

[0015] Optionally, the sensor is a camera, and the output information is an image captured by the camera; determining the first moment when the artificial eye moves to a preset position based on the output information includes:

[0016] The received image is detected, and the image in which the artificial eye moves to the preset position is determined as the target image;

[0017] The shooting time corresponding to the target image is determined as the first time.

[0018] Optionally, the method further includes:

[0019] Receive the output signal from the display screen, and determine the moment when the displayed information on the display screen changes as the third moment based on the output signal from the display screen.

[0020] The second delay time is determined based on the first time point and the third time point;

[0021] The second delay time is the time required from the occurrence of the eye movement event to the change of the displayed information on the display screen.

[0022] Optionally, when the XR device under test determines that the artificial eye has moved to the preset position through the eye-tracking module, it controls the brightness of the display screen to change; the display screen outputs a brightness value; the display screen is connected to an optical module through an optical fiber, the optical module being used to convert the optical signal into an analog electrical signal; the optical module is connected to an analog-to-digital converter module, the analog-to-digital converter being used to convert the analog electrical signal into a brightness value;

[0023] The third moment is determined by the moment when the displayed information on the screen changes based on the output signal of the display screen, including:

[0024] The moment when the brightness value changes is defined as the third moment.

[0025] In a second aspect, the present invention provides a delay time determination device, the device being used to determine the delay time of eye tracking in an XR device under test, wherein the eye tracking module in the XR device under test is used to detect eye movement events of an artificial eye; the device includes:

[0026] The first processing module is used to receive the output information from the sensor and determine the first moment when the artificial eye moves to a preset position based on the output information; the output information is used to represent the actual motion information of the artificial eye.

[0027] The second processing module is used to determine the moment when the synchronization signal is received as the second moment; the synchronization signal is the signal output by the XR device under test when the artificial eye moves to the preset position as determined by the eye-tracking module.

[0028] The determining module is used to determine a first delay time based on the first time and the second time.

[0029] Wherein, the first delay time is the delay from the occurrence of the eye movement event to the detection of the eye movement event by the XR device under test.

[0030] Thirdly, the present invention provides an electronic device, comprising: at least one processor and a memory;

[0031] The memory stores computer-executed instructions;

[0032] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any of the first aspects.

[0033] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in any of the first aspects.

[0034] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the first aspects.

[0035] This invention provides a method, apparatus, electronic device, and storage medium for determining delay time. The method is used to determine the delay time of eye tracking in an XR device under test. The eye tracking module in the XR device under test is used to detect eye movement events of an artificial eye. The method includes: receiving output information from a sensor; determining a first moment when the artificial eye moves to a preset position based on the output information, wherein the output information represents the actual movement information of the artificial eye; determining a second moment when a synchronization signal is received, wherein the synchronization signal is a signal output by the XR device under test when the eye tracking module determines that the artificial eye has moved to the preset position; determining a first delay time based on the first moment and the second moment, wherein the first delay time is the delay from the occurrence of an eye movement event to the detection of the eye movement event by the XR device under test; capturing the moment when the artificial eye moves to the preset position using a sensor; and capturing the moment when the synchronization signal generated when the eye tracking module determines that the artificial eye has moved to the preset position, thereby accurately determining the delay from the actual occurrence of the eye movement event to the detection of the eye movement event by the XR device under test through an algorithm, in order to determine the performance indicators of the XR device under test. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0037] Figure 1 An application scenario diagram provided by an embodiment of the present invention;

[0038] Figure 2 A flowchart illustrating a method for determining a delay time according to an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram illustrating the logical module function of an XR device under test, provided in an embodiment of the present invention.

[0040] Figure 4 This is a schematic diagram of the structure of a programmable artificial eye device provided in an embodiment of the present invention;

[0041] Figure 5 A schematic diagram illustrating simulated smooth eye tracking provided by an embodiment of the present invention;

[0042] Figure 6 An angular velocity curve of an artificial eye provided for an embodiment of the present invention;

[0043] Figure 7 This is a schematic diagram of the output signals of each phase of a motor encoder provided in an embodiment of the present invention;

[0044] Figure 8A flowchart illustrating another method for determining delay time provided in an embodiment of the present invention;

[0045] Figure 9 A schematic diagram of a hardware framework for determining a delay time provided in an embodiment of the present invention;

[0046] Figure 10 This is a schematic diagram of a delay time determination device provided in an embodiment of the present invention;

[0047] Figure 11 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention.

[0048] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0049] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention.

[0050] XR devices, including virtual reality devices, augmented reality devices, and mixed reality devices, typically incorporate eye-tracking modules to determine the user's eye movements in order to enable human-computer interaction or to render the user's gaze area in a high-quality manner, thereby enhancing the user experience.

[0051] For an XR device, the eye-tracking latency is an important performance indicator, and how to measure the eye-tracking latency of an XR device is a technical problem that needs to be solved.

[0052] Figure 1 This diagram illustrates an application scenario provided by an embodiment of the present invention. The method can be applied to a data processing platform, which can be an FPGA (Field-Programmable Gate Array) or a high-performance MCU (Microcontroller Unit). The data processing platform is mounted on a detection device used to detect the delay time of eye tracking in an XR device. Figure 1As shown, the data processing platform can receive the output information of the sensor corresponding to the artificial eye. The sensor is used to provide feedback on the actual movement information of the artificial eye. The XR device under test is equipped with an eye-tracking module. The XR device under test is worn on the artificial eye device so that the eye-tracking module in the XR device under test can detect eye movement events of the artificial eye. After detecting an eye movement event, the eye-tracking module of the XR device under test can generate a synchronization signal and transmit it to the data processing platform. After receiving the synchronization signal, the data processing platform can determine the first moment when the artificial eye moves to the preset position based on the sensor output information, determine the moment when the synchronization signal is received as the second moment, and determine the first delay time based on the first moment and the second moment, which is the delay from the occurrence of the eye movement event to the detection of the eye movement event by the XR device under test.

[0053] Figure 2 This is a flowchart illustrating a delay time determination method provided by an embodiment of the present invention. The method is used to determine the delay time of eye tracking in an XR device under test. The eye tracking module in the XR device under test is used to detect eye movement events of an artificial eye. The method includes steps S201 to S203:

[0054] Step S201: Receive the output information from the sensor, and determine the first moment when the artificial eye moves to the preset position based on the output information; the output information is used to represent the actual movement information of the artificial eye.

[0055] The artificial eye in this application can be a programmable artificial eye, meaning that its movement is controlled by a pre-programmed program. The artificial eye and the device used to control its movement constitute a programmable artificial eye device. This programmable artificial eye device can effectively simulate various human eye movements and its motion information can be captured by the eye-tracking module of an XR device. For example, simulated human eye movements can include rapid saccades and smooth tracking.

[0056] An eye-tracking module is installed in the XR device under test. The method of this application can be used to determine the delay from the occurrence of an eye movement event to the detection of the eye movement event by the eye-tracking module in the XR device under test, which is the theoretical delay of the eye-tracking algorithm.

[0057] Figure 3 This is a schematic diagram illustrating the logical module function of an XR device under test provided in an embodiment of the present invention, such as... Figure 3As shown, after determining the user's gaze point, the eye-tracking module in an XR device can execute corresponding user interaction logic, such as page turning or screen swiping. Furthermore, based on the gaze point determined by the eye-tracking module, gaze-point rendering compression can be performed, with high-resolution rendering applied to the gaze point location and low-resolution rendering applied to other locations. This significantly reduces the processing requirements of real-time rendering scenes while maintaining high frame rates and high-quality images, avoiding unnecessary rendering and reducing the load on the GPU (graphics processing unit). XR devices can also perform layer blending of the rendered image and the real image captured by the camera, and perform optical compensation on the blended image. During optical compensation, it can be based on the gaze point information to focus on improving the image quality at the gaze point location. After optical compensation, display compensation can also be performed, and the final processed information is displayed on the screen. Therefore, the eye-tracking module plays a crucial role in the XR device under test.

[0058] During the measurement, the XR device under test can be placed on the artificial eye platform so that the eye-tracking module can detect eye movement events of the artificial eye. Figure 4 This is a schematic diagram of the structure of a programmable artificial eye device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the XR device to be measured is placed on the artificial eye platform.

[0059] When measuring the delay time, the host computer can select an eye movement trajectory and choose a point on the trajectory as the preset position for this test. The artificial eye moves according to the selected eye movement trajectory. Sensors on the programmable artificial eye device can output the actual movement information of the artificial eye and transmit it to the data processing platform. The data processing platform receives this output information and determines the moment when the artificial eye reaches the preset position as the first moment. That is, the first moment is the moment when the artificial eye actually moves to the preset position.

[0060] Step S202: Determine the moment when the synchronization signal is received as the second moment; the synchronization signal is the signal output by the XR device under test when the artificial eye moves to the preset position as determined by the eye-tracking module.

[0061] The eye-tracking module in the XR device under test can detect eye movement events. Specifically, the eye-tracking module can acquire eye movement information. Based on the eye movement information, the XR device under test can determine whether the artificial eye has moved to a preset position. When it is determined that the artificial eye has moved to the preset position, the XR device under test can output a synchronization signal to the data processing platform. The data processing platform can determine the moment when the synchronization signal is received as the second moment.

[0062] In other words, the second moment is the moment when the XR device under test detects the eye movement event through hardware and algorithms. The eye tracking module consists of hardware and software algorithms. The hardware includes an infrared light source and an optical lens. The infrared light source is used to illuminate the eye so that the eye produces reflected light; the optical lens is used to capture the reflected light from the eye. The software algorithm is used to analyze the eye position and movement.

[0063] Step S203: Determine the first delay time based on the first time and the second time.

[0064] Wherein, the first delay time is the delay from the occurrence of the eye movement event to the detection of the eye movement event by the XR device under test.

[0065] Once the first and second moments are determined, the difference between the second and first moments can be calculated, and this difference is the first delay time.

[0066] For example, if the artificial eye moves to the preset position at the first moment of 8:05, and the second moment of the artificial eye moving to the preset position is determined by the eye-tracking module to be 8:06, then the first delay time is 1 minute.

[0067] This invention provides a method for determining delay time. The method is used to measure the delay time of eye tracking in an XR device under test. The eye tracking module in the XR device under test is used to detect eye movement events of an artificial eye. The method includes: receiving output information from a sensor; determining a first moment when the artificial eye moves to a preset position based on the output information, wherein the output information represents the actual movement information of the artificial eye; determining a second moment when a synchronization signal is received, wherein the synchronization signal is a signal output by the XR device under test when the eye tracking module determines that the artificial eye has moved to the preset position; determining a first delay time based on the first moment and the second moment, wherein the first delay time is the delay from the occurrence of an eye movement event to the detection of the eye movement event by the XR device under test; capturing the moment when the artificial eye moves to the preset position using a sensor; and capturing the moment when the synchronization signal generated when the eye tracking module determines that the artificial eye has moved to the preset position, thereby accurately determining the delay from the actual occurrence of the eye movement event to the detection of the eye movement event by the XR device under test through an algorithm, in order to determine the performance indicators of the XR device under test.

[0068] Optionally, the sensor is an inertial measurement unit, and the output information is acceleration and angular velocity at different times; determining the first moment when the artificial eye moves to the preset position based on the output information includes:

[0069] Based on the acceleration and angular velocity at different times, the first moment when the artificial eye moves to the preset position is determined.

[0070] Determining the first moment when the artificial eye reaches the preset position can be achieved using an Inertial Measurement Unit (IMU). The IMU can be mounted on the artificial eye; for example, it can be attached to the back of the artificial eye to avoid interfering with the eye-tracking module's eye tracking. When the artificial eye moves, the IMU can output acceleration and angular velocity at different times, and the initial position and initial velocity information can also be acquired each time the artificial eye is controlled to move. The data processing platform can then obtain the artificial eye's trajectory based on the acceleration and angular velocity at different times, thereby determining whether the artificial eye has reached the preset position, and defining the moment of reaching the preset position as the first moment.

[0071] Optionally, the data processing platform can perform integral calculations on acceleration and angular velocity to determine the position of the artificial eye.

[0072] Optionally, the preset position can be any position on the trajectory of the artificial eye's movement. Furthermore, there are no restrictions on the starting and ending positions of the artificial eye's movement.

[0073] The following is a specific implementation method for determining the first moment based on an inertial measurement unit. When the artificial eye is undergoing uniform acceleration and deceleration, the first moment when the artificial eye moves to the preset position can also be determined using the following method.

[0074] Optionally, the artificial eye moves within a range from a first field of view to a second field of view; the preset position is a field of view located at the midpoint between the first and second field of view; the movement of the artificial eye from the position of the first or second field of view to the preset position is a uniformly accelerated movement; the movement of the artificial eye from the preset position to the position of the first or second field of view is a uniformly decelerated movement.

[0075] Figure 5 This is a schematic diagram illustrating a simulated smooth eye-tracking method using an artificial eye, as provided in an embodiment of the present invention. Figure 5 As shown, when simulating eye movement, the artificial eye moves within a certain field of view, and the middle field of view within that field of view is determined as the preset position.

[0076] For example, when the artificial eye performs left and right scans, the field of view is a horizontal field of view, ranging from HFOV to ±α°. The preset position is HFOV = 0°, which is directly in front of the artificial eye. Therefore, the position where HFOV = -α° is the first field of view, and the position where HFOV = +α° is the second field of view. Here, it is set that when the artificial eye moves from HFOV = -α° to HFOV = 0°, it is uniformly accelerated; when it moves from HFOV = 0° to HFOV = α°, it is uniformly decelerated; when it moves from HFOV = +α° to HFOV = 0°, it is uniformly accelerated; and when it moves from HFOV = 0° to HFOV = -α°, it is uniformly decelerated.

[0077] Figure 6 An angular velocity curve of an artificial eye provided as an embodiment of the present invention, such as... Figure 6 As shown, when the artificial eye moves from HFOV = -α° to HFOV = 0°, the angular velocity gradually increases, and when it moves from HFOV = 0° to HFOV = α°, the angular velocity gradually decreases.

[0078] Optionally, determining the first moment when the artificial eye moves to the preset position based on the output information includes:

[0079] The moment when the angular velocity value is greater than the corresponding angular velocity value of the adjacent moment is determined as the first moment; or, the moment when the acceleration changes and the angular velocity value is not 0 is determined as the first moment.

[0080] When controlling the movement of the artificial eye, the motion from the position at the first or second field of view to the preset position is uniformly accelerated, and the motion from the preset position to the position at the first or second field of view is uniformly decelerated. Therefore, the angular velocity value of the artificial eye is always the largest when it reaches the preset position. Thus, the data processing platform can determine the angular velocity values ​​of the artificial eye at different times, and define the moment when the angular velocity value is greater than the corresponding angular velocity value at the adjacent moment as the first moment. Alternatively, the data processing platform can also define the moment when the acceleration information changes and the angular velocity value is not zero as the first moment.

[0081] By using the above methods to perform uniform acceleration and deceleration control on the artificial eye, the control of the artificial eye becomes relatively simple and easy to implement.

[0082] In addition to controlling the artificial eye to perform uniform acceleration and deceleration movements as described above, it is also possible to control the artificial eye to perform non-uniform acceleration or non-uniform deceleration movements. When performing such movements, the first field of view and the second field of view do not need to be set symmetrically.

[0083] Optionally, a table mapping the angular velocity of the artificial eye to its position can be pre-set. The first moment when the artificial eye reaches the preset position can be determined by querying this table. The angular velocity corresponding to the preset position can be determined by querying the table. The determined angular velocity is compared with the currently detected actual angular velocity. If it is within the error range, the artificial eye is determined to have reached the preset position.

[0084] Optionally, a standard motion trajectory and a preset position can be pre-set to control the artificial eye to move according to the standard motion trajectory. During the movement of the artificial eye, the position information of the artificial eye can be calculated based on the acceleration and angular velocity at different times output by the IMU. Each position information corresponds to a timestamp, which is the timestamp when the IMU receives the corresponding acceleration and angular velocity. Based on the position information of the artificial eye, the actual motion trajectory of the artificial eye can be obtained, and the timestamp corresponding to the preset position in the actual motion trajectory is determined as the first moment.

[0085] Furthermore, this application is not limited to the two methods mentioned above for determining the first moment when the artificial eye moves to the preset position; it can also be achieved based on other methods.

[0086] By using the output information from the inertial measurement unit, the motion trajectory of the artificial eye can be accurately determined, thereby determining the moment when the artificial eye moves to the preset position.

[0087] Optionally, the sensor is a motor encoder, and the output information is A-phase and B-phase pulses at different times; determining the first moment when the artificial eye moves to the preset position based on the output information includes:

[0088] The first moment when the artificial eye moves to the preset position is determined based on the number or phase relationship of pulses at different times.

[0089] The determination of the first moment when the artificial eye moves to the preset position can also be achieved based on a motor encoder.

[0090] The programmable artificial eye moves under the control of a motor, with a motor encoder mounted on the motor to provide closed-loop feedback signals. When the artificial eye undergoes non-uniform acceleration or deceleration, the first moment when the artificial eye reaches the preset position can be determined based on the number of pulses at different times.

[0091] Optionally, the motor encoder output can be either high or low level, and the data processing platform can obtain the number of pulses based on the information output by the motor encoder. Specifically, the artificial eye can be controlled to move along a pre-set standard motion trajectory and a preset position. During the movement of the artificial eye, the position information of the artificial eye can be calculated based on the number of pulses output by the motor encoder at different times. Each position information corresponds to a timestamp, which is the timestamp when the IMU receives the corresponding number of pulses. Based on the position information of the artificial eye, the actual motion trajectory of the artificial eye can be obtained, and the timestamp corresponding to the preset position in the actual motion trajectory is determined as the first moment.

[0092] The following describes a specific implementation method for determining the first moment based on a motor encoder. When the artificial eye performs uniform acceleration and deceleration, the phase relationship of pulses at different moments can also be used to determine the first moment when the artificial eye moves to the preset position.

[0093] Optionally, the artificial eye moves within a range from a first field of view to a second field of view; the preset position is a field of view located at the midpoint between the first and second field of view; the movement of the artificial eye from the position of the first or second field of view to the preset position is a uniformly accelerated movement; the movement of the artificial eye from the preset position to the position of the first or second field of view is a uniformly decelerated movement.

[0094] Optionally, determining the first moment when the artificial eye moves to the preset position based on the phase relationship of pulses at different times includes:

[0095] Based on the phase relationship of pulses at different times, the two adjacent times at which the lead information changes are determined, and the first time is determined based on the two adjacent times at which the lead information changes; the lead information represents the relationship between the pulse signal of phase A and the pulse signal of phase B.

[0096] The motor encoder is an incremental encoder. It is assumed that the artificial eye moves in the CW direction when moving from HFOV = -α° to HFOV = +α°, and in the CCW direction when moving from HFOV = +α° to HFOV = -α°. Figure 7 This is a schematic diagram of the output signals of each phase of a motor encoder provided in an embodiment of the present invention, as shown below. Figure 7As shown, when rotating in the CW direction, the high-level output of phase A leads phase B; when rotating in the CCW direction, the high-level output of phase A lags phase B, meaning the high-level output of phase B leads phase A. Therefore, when the artificial eye moves to HFOV = +α° or HFOV = -α°, the lead information changes. Thus, based on the pulse output signals at different times, the moment when the lead information changes can be determined, and the two adjacent moments of the lead information change can be recorded. The first moment can be determined based on the two adjacent moments of the lead information change. Optionally, the two adjacent moments of the lead information change can be added together, and the result of the addition divided by 2 is determined as the first moment.

[0097] For example, when the artificial eye moves to HFOV = -α°, the leading positions of phases A and B change, and the data processing platform records this as time t1; when the artificial eye moves to HFOV = α°, the leading positions of phases A and B change again, and the data processing platform records this as time t2. Because the artificial eye performs uniform acceleration and deceleration motion, the time when the eyeball moves to HFOV = 0° is (t1 + t2) / 2, which is the first moment. By using the above method to perform uniform acceleration and deceleration control on the artificial eye, the control of the artificial eye becomes relatively simple and easy to implement.

[0098] By using the output information from the motor encoder, the motion trajectory of the artificial eye can be accurately determined, thereby determining the moment when the artificial eye moves to the preset position.

[0099] Optionally, the sensor is a camera, and the output information is an image captured by the camera; determining the first moment when the artificial eye moves to a preset position based on the output information includes:

[0100] The received image is detected, and the image in which the artificial eye moves to the preset position is determined as the target image; the shooting time corresponding to the target image is determined as the first time.

[0101] In addition, the sensor can also be a camera. While controlling the movement of the artificial eye, the camera can be triggered to work at the same time. The camera can continuously take pictures of the artificial eye while it is moving. By judging each picture of the artificial eye, it can be determined whether the artificial eye in the picture has moved to a preset position. Once the target picture of the artificial eye moving to the preset position is determined, the time of the capture of the target picture can be determined.

[0102] Optionally, the camera assigns a timestamp to each image it captures. The data processing platform can receive each image captured by the camera and its corresponding capture time, thus determining the capture time of the target image after identifying it. Alternatively, the data processing platform can control the camera to capture images, allowing it to directly obtain the capture time of each image, thereby determining the capture time of the target image after identifying it.

[0103] Based on the camera's output information, the motion trajectory of the artificial eye can be accurately determined, thereby determining the moment when the artificial eye moves to the preset position.

[0104] Optionally, the method further includes:

[0105] Receive the output signal from the display screen, and determine the moment when the displayed information on the display screen changes as the third moment based on the output signal from the display screen.

[0106] The second delay time is determined based on the first time point and the third time point;

[0107] The second delay time is the time required from the occurrence of the eye movement event to the change of the displayed information on the display screen.

[0108] In addition to the first delay time, a second delay time can also be measured, namely the end-to-end delay of eye tracking, which is the time required from the actual occurrence of an eye-tracking event to the display of the corresponding change on the screen of the XR device under test.

[0109] Typically, when an XR device detects an eye-tracking event using its eye-tracking module, it can control the display to perform corresponding actions. After an eye-tracking event occurs, the screen display will change accordingly; for example, when a user looks at an icon, the XR device can zoom in on that image.

[0110] The second latency is added to the first latency by incorporating delays in Unity CPU (Central Processing Unit) rendering, Unity GPU rendering, runtime rendering, and display rendering. The second latency is also known as MTP (Motion to Photon) latency, which is the time difference between a change in the user's eye gaze direction and a change appearing on the terminal's display.

[0111] The second latency is also important for XR devices. If the second latency is too long, it will affect the user experience. Therefore, it is necessary to measure the second latency.

[0112] Optionally, the data processing platform can receive the output signal from the display screen of the XR device under test, determine the moment when the displayed information on the screen changes based on the output signal, and designate this moment as the third moment. Specifically, a change in the displayed information can be triggered when the artificial eye is detected to move to a preset position.

[0113] After determining the third time point, the difference between the third time point and the first time point can be determined as the second delay time.

[0114] For example, if the artificial eye moves to the preset position at the first moment of 8:05 and the third moment when it is determined that the information displayed on the screen has changed is 8:07, then the second delay time is 2 minutes.

[0115] The eye-tracking module determines when the artificial eye moves to a preset position, controls the change in the displayed information on the screen, and determines the second delay time based on the actual time when the displayed information on the screen changes, so as to accurately measure the MTP delay.

[0116] Optionally, when the XR device under test determines that the artificial eye has moved to the preset position through the eye-tracking module, it controls the brightness of the display screen to change; the display screen outputs a brightness value; the display screen is connected to an optical module through an optical fiber, the optical module being used to convert the optical signal into an analog electrical signal; the optical module is connected to an analog-to-digital converter module, the analog-to-digital converter being used to convert the analog electrical signal into a brightness value;

[0117] The third moment is determined by the moment when the displayed information on the screen changes based on the output signal of the display screen, including:

[0118] The moment when the brightness value changes is defined as the third moment.

[0119] The XR device under test uses an eye-tracking module to determine when the artificial eye moves to a preset position, and then controls the brightness of the display screen to change accordingly.

[0120] For example, such as Figure 5 As shown, when HFOV ≤ 0°, the XR device screen displays black; when HFOV > 0°, the XR device screen displays white. The optical module converts the optical signal into an analog electrical signal, which is then converted into a digital signal (brightness value) by the analog-to-digital converter (ADC module) and transmitted to the MCU. The MCU receives the brightness value and determines the moment when the brightness value changes as the third moment.

[0121] By controlling the brightness of the display screen to change when the artificial eye moves to a preset position, it is easier to determine when the displayed information on the screen changes.

[0122] Figure 8 This is a flowchart illustrating another delay time determination method provided in an embodiment of the present invention. It is assumed that the motion trajectory of the artificial eye is trajectory A; the motion trajectory of the artificial eye calculated from the sensors is trajectory B; and the motion trajectory of the artificial eye calculated from the eye movement information obtained by the eye-tracking module is trajectory C. The data processing platform is MCU as an example for illustration.

[0123] like Figure 8 As shown, when determining the delay time of the eye-tracking module, the host computer can select eye movement trajectory A and choose a point in trajectory A as the preset position for this test. The programmable artificial eye device controls the movement of the artificial eye according to the selected trajectory A. The MCU determines whether the eye has moved to the preset position based on trajectory B. If so, the MCU generates a timestamp T1. At the same time, the eye-tracking module in the XR device works and outputs eye movement information. The XR device determines whether the eye has moved to the preset position based on the eye movement information and trajectory C. If so, the XR device outputs a synchronization signal. When the MCU detects the synchronization signal, it generates a timestamp T2. In addition, the XR device can also perform different applications based on the eye movement information, that is, control the display screen to perform corresponding operations. The screen of the XR device has an output signal. After the MCU detects the output signal of the screen and determines that the screen has performed the corresponding operation, it generates a timestamp T3. Finally, based on T2 and T1, the first delay time is obtained, which is the theoretical delay of the eye-tracking algorithm. Based on T3 and T1, the second delay time is obtained, which is the end-to-end delay of eye tracking.

[0124] Figure 9 This is a schematic diagram of a hardware framework for determining a delay time, provided in an embodiment of the present invention. The data processing platform is MCU (Microcontroller Unit) as an example for illustration. Figure 9As shown, the MCU (data processing platform) includes an eye movement detection module, which determines whether the artificial eye has moved to a preset position based on sensor output information. A timestamp generation module generates a timestamp T1 when the artificial eye reaches the preset position. The XR device under test contains a synchronization signal, i.e., a synchronization signal generation module. Triggered by the eye-tracking module, it generates a synchronization signal and transmits it to the MCU's synchronization signal processing module. The timestamp module generates a timestamp T2 when it receives the synchronization signal. The XR device also includes a display screen, which is connected to an optical module (PD module) via fiber optic cable. The output signal of the optical module is converted by an analog-to-digital converter (ADC) to obtain the display screen's output signal, which is then transmitted to the PD data processing module in the MCU. After determining the appropriate operation based on the display screen's output signal, the PD data processing module generates a timestamp T3. Two delay times are calculated based on these three timestamps. The MCU can also include a data transmission module to send the detected data to a PC for processing to calculate the two delay times.

[0125] Figure 10 This is a schematic diagram of a delay time determination device provided in an embodiment of the present invention. The device is used to determine the delay time of eye tracking in an XR device under test. The eye tracking module in the XR device under test is used to detect eye movement events of an artificial eye. The device 100 includes:

[0126] The first processing module 1001 is used to receive the output information from the sensor and determine the first moment when the artificial eye moves to a preset position based on the output information; the output information is used to represent the actual motion information of the artificial eye.

[0127] The second processing module 1002 is used to determine the moment when the synchronization signal is received as the second moment; the synchronization signal is the signal output by the XR device under test when the artificial eye moves to the preset position as determined by the eye-tracking module.

[0128] The first determining module 1003 is used to determine a first delay time based on the first time and the second time.

[0129] Wherein, the first delay time is the delay from the occurrence of the eye movement event to the detection of the eye movement event by the XR device under test.

[0130] Optionally, the sensor is an inertial measurement unit, and the output information is acceleration and angular velocity at different times; when the first processing module 1001 determines the first moment when the artificial eye moves to the preset position based on the output information, it is specifically used for:

[0131] Based on the acceleration and angular velocity at different times, the first moment when the artificial eye moves to the preset position is determined.

[0132] Optionally, the sensor is a motor encoder, and the output information is A-phase and B-phase pulses at different times; when the first processing module 1001 determines the first moment when the artificial eye moves to the preset position based on the output information, it is specifically used for:

[0133] The first moment when the artificial eye moves to the preset position is determined based on the number or phase relationship of pulses at different times.

[0134] Optionally, the sensor is a camera, and the output information is an image captured by the camera; when the first processing module 1001 determines the first moment when the artificial eye moves to the preset position based on the output information, it is specifically used for:

[0135] The received image is detected, and the image in which the artificial eye moves to the preset position is determined as the target image;

[0136] The shooting time corresponding to the target image is determined as the first time.

[0137] Optionally, the device further includes:

[0138] The receiving module is used to receive the output signal from the display screen and determine the time when the displayed information on the display screen changes as the third moment based on the output signal from the display screen.

[0139] The second determining module is used to determine the second delay time based on the first time and the third time.

[0140] The second delay time is the time required from the occurrence of the eye movement event to the change of the displayed information on the display screen.

[0141] Optionally, when the XR device under test determines that the artificial eye has moved to the preset position through the eye-tracking module, it controls the brightness of the display screen to change; the display screen outputs a brightness value; the display screen is connected to an optical module through an optical fiber, the optical module being used to convert the optical signal into an analog electrical signal; the optical module is connected to an analog-to-digital converter module, the analog-to-digital converter being used to convert the analog electrical signal into a brightness value;

[0142] When the receiving module determines the moment when the displayed information on the screen changes based on the output signal of the display screen as the third moment, it is specifically used for:

[0143] The moment when the brightness value changes is defined as the third moment.

[0144] The delay time determination device 100 provided in this embodiment of the invention can achieve the above-mentioned... Figure 2 The delay time determination method in the illustrated embodiment has a similar implementation principle and technical effect, and will not be described again here.

[0145] Figure 11 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Figure 11 As shown, the electronic device provided in this embodiment includes at least one processor 1101 and a memory 1102. The processor 1101 and the memory 1102 are connected via a bus 1103.

[0146] In a specific implementation, at least one processor 1101 executes computer execution instructions stored in memory 1102, causing at least one processor 1101 to execute the method in the above method embodiment.

[0147] The specific implementation process of processor 1101 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0148] In the above Figure 11 In the illustrated embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0149] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0150] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0151] This invention also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in the above embodiments.

[0152] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the above method embodiments.

[0153] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0154] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0155] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0156] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0158] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for determining a delay time, characterized in that, The method is used to determine the delay time of eye tracking in an XR device under test, wherein the eye tracking module in the XR device under test is used to detect eye movement events of the artificial eye; the method includes: The system receives output information from the sensor and determines the first moment when the artificial eye moves to a preset position based on the output information; the output information is used to represent the actual motion information of the artificial eye. The moment when the synchronization signal is received is defined as the second moment; the synchronization signal is the signal output by the XR device under test when the artificial eye moves to the preset position, as determined by the eye-tracking module. The first delay time is determined based on the first time point and the second time point; Wherein, the first delay time is the delay from the occurrence of the eye movement event to the detection of the eye movement event by the XR device under test.

2. The method according to claim 1, characterized in that, The sensor is an inertial measurement unit, and the output information is the acceleration and angular velocity at different times; Determining the first moment when the artificial eye moves to the preset position based on the output information includes: Based on the acceleration and angular velocity at different times, the first moment when the artificial eye moves to the preset position is determined.

3. The method according to claim 1, characterized in that, The sensor is a motor encoder, and the output information is A-phase and B-phase pulses at different times; determining the first moment when the artificial eye moves to the preset position based on the output information includes: The first moment when the artificial eye moves to the preset position is determined based on the number or phase relationship of pulses at different times.

4. The method according to claim 1, characterized in that, The sensor is a camera, and the output information is an image captured by the camera; Determining the first moment when the artificial eye moves to the preset position based on the output information includes: The received image is detected, and the image in which the artificial eye moves to the preset position is determined as the target image; The shooting time corresponding to the target image is determined as the first time.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Receive the output signal from the display screen, and determine the moment when the displayed information on the display screen changes as the third moment based on the output signal from the display screen. The second delay time is determined based on the first time point and the third time point; The second delay time is the time required from the occurrence of the eye movement event to the change of the displayed information on the display screen.

6. The method according to claim 5, characterized in that, The XR device under test determines when the artificial eye moves to the preset position through the eye-tracking module, and controls the brightness of the display screen to change; the display screen outputs a brightness value; the display screen is connected to an optical module through an optical fiber, and the optical module is used to convert the optical signal into an analog electrical signal; the optical module is connected to an analog-to-digital converter module, and the analog-to-digital converter is used to convert the analog electrical signal into a brightness value; The third moment is determined by the moment when the displayed information on the screen changes based on the output signal of the display screen, including: The moment when the brightness value changes is defined as the third moment.

7. A delay time determination device, characterized in that, The device is used to measure the delay time of eye tracking in the XR device under test, and the eye tracking module in the XR device under test is used to detect eye movement events of the artificial eye; the device includes: The first processing module is used to receive the output information from the sensor and determine the first moment when the artificial eye moves to a preset position based on the output information; the output information is used to represent the actual motion information of the artificial eye. The second processing module is used to determine the moment when the synchronization signal is received as the second moment; the synchronization signal is the signal output by the XR device under test when the artificial eye moves to the preset position as determined by the eye-tracking module. The determining module is used to determine a first delay time based on the first time and the second time. Wherein, the first delay time is the delay from the occurrence of the eye movement event to the detection of the eye movement event by the XR device under test.

8. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images