Delay Testing Method, Delay Detection Device, Electronic Device and Storage Medium

Abstract

The present disclosure provides a delay testing method, a delay detection device, an electronic device, and a storage medium, which specifically relate to the technical field of delay testing. The delay testing method includes: in response to receiving a first action sequence, obtaining a first motion trajectory of a simulation component in a preset time period according to the first action sequence, where the first action sequence includes first time information and first pose information; obtaining a second action sequence according to a display image of the display device during the movement of the simulation component according to the first motion trajectory, and determining a second motion trajectory in the preset time period according to the second action sequence, where the second action sequence includes second time information and second pose information; and determining the delay of the display device in the preset time period based on the first motion trajectory and the second motion trajectory.

Description

Technical Field

[0001] The present disclosure relates to the technical field of latency testing, and particularly to a latency testing method, a latency detection device, an electronic device, and a storage medium. Background Art

[0002] With the development of science and technology, higher requirements are put forward for the performance of display devices such as virtual reality devices, augmented reality devices, and mixed reality devices. MTP (motion-to-photon) latency refers to the time taken from the start of a user's movement to the corresponding image being displayed on the screen, which is an important parameter reflecting the performance of a display device. For users, the smaller the MTP latency, the better the user's sense of immersion; the larger the MTP latency, the stronger the user's sense of dizziness.

[0003] The test of MTP latency is a process that needs to be carried out before the display device leaves the factory. The test accuracy of MTP latency is relatively important, which can guide R & D personnel to better improve the display device, thereby reducing the MTP latency of the display device. However, the test accuracy of the MTP latency measured by using the existing test method is low. Summary of the Invention

[0004] The first aspect of the present disclosure provides a latency testing method, which includes: in response to receiving a first action sequence, obtaining a first motion trajectory of a simulation component in a preset time period according to the first action sequence, where the first action sequence includes first time information and first pose information; obtaining a second action sequence according to the display image of the display device during the movement of the simulation component along the first motion trajectory, and determining a second motion trajectory in the preset time period according to the second action sequence, where the second action sequence includes second time information and second pose information; and determining the latency of the display device in the preset time period based on the first motion trajectory and the second motion trajectory.

[0005] In some embodiments, obtaining a first motion trajectory of a simulation component in a preset time period according to the first action sequence includes: in response to the existence of a target action sequence corresponding to the first action sequence in a database, determining the target motion trajectory corresponding to the target action sequence as the first motion trajectory.

[0006] In some embodiments, obtaining a first motion trajectory of a simulation component in a preset time period according to the first action sequence includes: in response to the non-existence of a target action sequence corresponding to the first action sequence in a database, obtaining the pose information of a reference point at different time points during the user performing the action corresponding to the first action sequence; and determining the first motion trajectory based on the pose information of the reference point at different time points.

[0007] In some embodiments, obtaining the pose information of a reference point at different time points during a user performing an action corresponding to a first action sequence includes: obtaining the pose information of a plurality of points to be measured at different time points during a user performing an action corresponding to the first action sequence; and determining the pose information of the reference point at different time points according to the pose information of the plurality of points to be measured at different time points, where the reference point is the center point of the three-dimensional space formed by the plurality of points to be measured.

[0008] In some embodiments, determining a first motion trajectory based on the pose information of the reference point at different time points includes: determining the pose information of a simulation component at different time points according to the pose information of the reference point at different time points; determining the motion parameters of the simulation component at different time points according to the pose information of the simulation component at different time points, where the motion parameters include one or more of the rotation angle, speed, and acceleration of the simulation component; and generating a third motion trajectory based on the pose information and motion parameters of the simulation component at different time points, and determining the third motion trajectory as the first motion trajectory.

[0009] In some embodiments, obtaining a second action sequence according to the display image of the display device during the simulation component moving according to the first motion trajectory includes: performing feature extraction on the display image to obtain feature points, and determining the two-dimensional coordinates of the feature points at different time points; determining the three-dimensional coordinates of the feature points at different time points according to the identity identifiers of the feature points, and constructing two-dimensional coordinate - three-dimensional coordinate matching point pairs of the feature points at different time points; and determining the second action sequence based on the two-dimensional coordinate - three-dimensional coordinate matching point pairs of the feature points at different time points.

[0010] In some embodiments, determining the second action sequence based on the two-dimensional coordinate - three-dimensional coordinate matching point pairs of the feature points at different time points includes: performing pose calculation on the two-dimensional coordinate - three-dimensional coordinate matching point pairs of the feature points at different time points to obtain the rotation matrix of the display image relative to the world coordinate system at different time points; and determining the second action sequence based on the rotation matrices at different time points.

[0011] In some embodiments, before performing feature extraction on the display image to obtain feature points, it further includes: performing Gaussian blur processing on the display image.

[0012] In some embodiments, determining the delay of the display device in a preset time period based on the first motion trajectory and the second motion trajectory includes: performing linear fitting and / or quadratic curve fitting on the first motion trajectory and the second motion trajectory; and determining the delay of the display device in the preset time period according to the first motion trajectory and the second motion trajectory after fitting.

[0013] In some embodiments, performing linear fitting and / or quadratic curve fitting on the first motion trajectory and the second motion trajectory includes: identifying the linear intervals and non-linear intervals of the first motion trajectory and the second motion trajectory; and performing linear fitting on the linear intervals and quadratic curve fitting on the non-linear intervals.

[0014] In some embodiments, the first motion trajectory and the second motion trajectory are time-pose curves. Determining the delay of the display device in a preset time period according to the first motion trajectory and the second motion trajectory after fitting includes: obtaining a delay curve of the display device according to the difference in time between the second motion trajectory and the first motion trajectory after fitting at the same pose; and determining the delay of the display device in the preset time period based on the delay curve of the display device.

[0015] In some embodiments, the time-pose curve is a time-rotation angle curve. Obtaining a delay curve of the display device according to the difference in time between the second motion trajectory and the first motion trajectory after fitting at the same pose includes: obtaining a delay curve of the display device according to the difference in time between the second motion trajectory and the first motion trajectory after fitting at the same rotation angle, where the rotation angle includes roll angle, yaw angle, and pitch angle.

[0016] A second aspect of the present disclosure provides a delay detection device, which includes an analog component, a driving motor, a camera module, and a controller. The analog component is used to simulate the actions of a user. The driving motor is used to drive the analog component to move along a first motion trajectory, where the first motion trajectory is determined based on a first action sequence including first time information and first pose information. The camera module is used to collect display images of the display device to be tested during the movement of the analog component along the first motion trajectory. The controller is communicatively connected to the driving motor and the camera module, where the controller is configured to: determine a second motion trajectory in a preset time period according to the display images, and determine the delay of the display device to be tested in the preset time period based on the first motion trajectory and the second motion trajectory.

[0017] A third aspect of the present disclosure provides an electronic device, which includes at least one processor and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the delay test method of the first aspect of the present disclosure.

[0018] A fourth aspect of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the delay test method as in the first aspect of the present disclosure.

[0019] The present disclosure obtains a first motion trajectory of a simulation component in a preset time period through a first action sequence. The simulation component can simulate the actions of different users, and make the actions of the simulation component more realistic. Then, during the movement of the simulation component according to the first motion trajectory, a second motion trajectory in the preset time period is determined according to a second action sequence obtained from the display image of the display device, and the delay of the display device in the preset time period is determined according to the first motion trajectory and the second motion trajectory, realizing the delay test in the preset time period and improving the test accuracy of the delay of the display device.

[0020] It should be understood that the content described in this part is not intended to identify the key features or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Brief Description of the Drawings

[0021] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects, and advantages involved in the embodiments of the present disclosure will become more obvious. Among them:

[0022] Figure 1 Shows an exemplary system framework applicable to the delay test method according to the present disclosure;

[0023] Figure 2 Shows another exemplary system framework applicable to the delay test method according to the present disclosure;

[0024] Figure 3 Shows a flowchart of the delay test method according to an exemplary embodiment of the present disclosure;

[0025] Figure 4 Shows a schematic diagram of the attitude angle according to an exemplary embodiment of the present disclosure;

[0026] Figure 5 Shows a schematic diagram of the distribution of the points to be measured according to an exemplary embodiment of the present disclosure;

[0027] Figure 6 Shows a schematic diagram of the first motion trajectory and the second motion trajectory according to an exemplary embodiment of the present disclosure;

[0028] Figure 7 Shows the Figure 6 Schematic diagram of linear fitting of the first motion trajectory in the region G in;

[0029] Figure 8 Shows the Figure 6 Schematic diagram of linear fitting of the second motion trajectory in the region G in;

[0030] Figure 9 A schematic diagram showing the first and second fitted motion trajectories according to an exemplary embodiment of the present disclosure;

[0031] Figure 10 A schematic diagram showing the delay curve of a display device according to an exemplary embodiment of the present disclosure;

[0032] Figure 11 A schematic diagram showing the interaction between a display device and a delay detection device according to an exemplary embodiment of the present disclosure;

[0033] Figure 12 A schematic diagram showing the interaction between a display device, a delay detection device, and a server according to an exemplary embodiment of the present disclosure;

[0034] Figure 13 A schematic diagram showing the structure of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Embodiments

[0035] To better understand the present disclosure, more detailed descriptions of various aspects of the present disclosure will be made with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present disclosure and do not limit the scope of the present disclosure in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0036] It should be noted that in this specification, the expressions such as "first", "second", etc. are only used to distinguish one feature from another feature and do not represent any limitation on the feature. Therefore, without departing from the teachings of the present disclosure, the first motion trajectory discussed below may also be referred to as the second motion trajectory, and the second motion trajectory may also be referred to as the first motion trajectory.

[0037] It should also be understood that expressions such as "comprising", "including", "having", "containing", and / or "including having" are open-ended rather than closed-ended expressions in this specification, which means that there are the stated features, elements, and / or components, but do not exclude the existence of one or more other features, elements, components, and / or their combinations. Additionally, "exemplarily" is used to refer to an example or illustration.

[0038] Unless otherwise defined, all terms (including technical terms and scientific terms) used in the present disclosure have the same meaning as commonly understood by those of ordinary skill in the art to which the present disclosure pertains. It should also be understood that terms (such as those defined in a common dictionary) should be understood to have a meaning consistent with their meaning in the context of the relevant art and should not be interpreted in an idealized or overly formal sense unless clearly defined as such in the present disclosure.

[0039] It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments may be combined with each other. Additionally, unless explicitly defined or contradictory to the context, the specific steps included in the methods described in the present disclosure do not have to be limited to the recorded order, but can be executed in any order or executed in parallel. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0040] Figure 1 and Figure 2 respectively show an exemplary system architecture 100 of a delay test method that can be applied to some embodiments of the present disclosure.

[0041] In an exemplary embodiment, as Figure 1 shown, the system architecture 100 may include a display device 110 and a delay detection device 120. The display device 110 may include one or more of a virtual reality device, an augmented reality device, and a mixed reality device. The delay detection device 120 may include an analog component 121, a controller 122, a drive motor 123, and a camera module 124. The analog component 121 may be used to simulate the actions of a user. The drive motor 123 may be used to drive the analog component 121 to move. The camera module 124 may be used to collect the display images of the display device 110 during the movement of the analog component 121. The controller 122 may be communicatively connected to the drive motor 123 and the camera module 124, for example. The controller 122 may be communicatively connected to the display device 110, for example.

[0042] It should be noted that the delay test method of the exemplary embodiment of the present disclosure may be executed by the controller 122 of the delay detection device 120.

[0043] In an exemplary embodiment, as Figure 2As shown, the system architecture 100 may include a display device 110, a delay detection device 120, and a server 130. The display device 110 may include one or more of a virtual reality device, an augmented reality device, and a mixed reality device. The delay detection device 120 may include an analog component 121, a controller 122, a drive motor 123, and a camera module 124. The analog component 121 may be used to simulate the actions of a user. The drive motor 123 may be used to drive the analog component 121 to move. The camera module 124 may be used to collect the display images of the display device 110 during the movement of the analog component 121. The controller 122 may be communicatively connected to the drive motor 123 and the camera module 124, for example. The server 130 may be communicatively connected to the display device 110 and the controller 122, for example. The server 130 may provide various services based on the display device 110 and the delay detection device 120. For example, the server 130 may analyze and process the received first action sequence and generate a processing result (for example, obtain the first movement trajectory of the analog component 121 according to the first action sequence).

[0044] It should be noted that the server 130 may be hardware or software. When the server 130 is hardware, it may be implemented as a distributed server cluster composed of multiple servers or as a single server. When the server 130 is software, it may be implemented as multiple software or software modules (for example, to provide distributed services) or as a single software or software module. No specific limitation is made here. In addition, the delay test method of the exemplary embodiment of the present disclosure may be executed by the server 130.

[0045] Figure 3 is a schematic flowchart of a delay test method 200 according to an exemplary embodiment of the present disclosure. The delay test method 200 may be executed by, for example, the controller 122 of the delay detection device 120 (such as Figure 1 ) or the server 130 (such as Figure 2 ) and other electronic devices. It should be understood that the delay test method 200 may further include additional steps not shown and / or steps shown may be omitted, and the scope of the present disclosure is not limited in this regard. The delay test method 200 may be applicable to application scenarios such as the delay test of the display device 110. Among them, the display device 110 may include a virtual reality device, an augmented reality device, or a mixed reality device. It should be understood that the delay test method 200 may also be applied to other application scenarios of the delay test of display devices, and no limitation is made here.

[0046] As Figure 3 shown, the delay test method 200 may include the following steps:

[0047] S210. In response to receiving the first action sequence, obtain the first motion trajectory of the simulation component in a preset time period according to the first action sequence, where the first action sequence includes first time information and first pose information.

[0048] S220. Obtain a second action sequence according to the display image of the display device during the movement of the simulation component along the first motion trajectory, and determine the second motion trajectory in the preset time period according to the second action sequence, where the second action sequence includes second time information and second pose information.

[0049] S230. Determine the delay of the display device in the preset time period based on the first motion trajectory and the second motion trajectory.

[0050] The method provided by the exemplary embodiment of the present disclosure obtains the first motion trajectory of the simulation component in a preset time period through the first action sequence, can simulate the actions of different users by using the simulation component, and makes the actions of the simulation component more realistic; then, during the movement of the simulation component along the first motion trajectory, determine the second motion trajectory in the preset time period according to the second action sequence obtained from the display image of the display device, and the delay of the display device in the preset time period can be determined according to the first motion trajectory and the second motion trajectory, realizing the delay test in the preset time period and improving the test accuracy of the delay of the display device.

[0051] The steps S210 to S230 of the exemplary embodiment of the present disclosure will be described in detail below.

[0052] In step S210, in response to receiving the first action sequence, obtain the first motion trajectory of the simulation component in a preset time period according to the first action sequence, where the first action sequence may include first time information and first pose information.

[0053] In the exemplary embodiment, the first action sequence may be an action sequence corresponding to the action to be measured. For example, the first action sequence may be an action sequence constructed by the action parameters at different time points during the user's action according to the action to be measured. The number of action parameters in the first action sequence can be determined by the action execution time of the action to be measured and the sampling time interval. The sampling time interval between two adjacent action parameters in the first action sequence may be the same or different. For example, the action execution time of the action to be measured is 1 s, the sampling time interval is 0.01 s, and the total number of samples from the start time to the end time of the execution of the action to be measured is 101, that is, the number of action parameters in the first action sequence is 101.

[0054] The action parameters in the first action sequence can be represented by "t m _roll m _pitch m_yaw m in the form of “t m m_roll m _pitch m _yaw m ”, where 1 ≤ m ≤ a, a is the number of motion parameters in the first motion sequence, t m is the time corresponding to the m-th motion parameter in the first motion sequence, roll m _pitch m _yaw m m is the attitude angle corresponding to the m-th motion parameter in the first motion sequence. As shown in Figure 4 , in the world coordinate system, roll is the angle of rotation around the Z-axis (i.e., the roll angle), pitch is the angle of rotation around the X-axis (i.e., the pitch angle), and yaw is the angle of rotation around the Y-axis (i.e., the yaw angle). The above t m m _pitch m _yaw m is the first time information of the first motion sequence, and the above roll m _pitch m _yaw m Figure 4 is the first pose information of the first motion sequence. m For example, the number of motion parameters in the first motion sequence can be 101. The first motion parameter in the first motion sequence can be “t1_roll1_pitch1_yaw1”, where t1 is the starting moment of performing the action to be measured, and roll1_pitch1_yaw1 is the attitude angle corresponding to the starting moment. “t1_roll1_pitch1_yaw1” can be, for example, “000_10_20_30”, where “000” represents the starting moment, and “10_20_30” represents that the roll angle at the starting moment is 10°, the pitch angle is 20°, and the yaw angle is 30°. The one-hundred-and-first motion parameter in the first motion sequence can be “t 101 _roll 101 _pitch 101 _yaw 101 m _pitch m _yaw m ”, where t 101 is the ending moment of performing the action to be measured, and roll 101 _pitch 101 _yaw 101

[0055] is the attitude angle corresponding to the ending moment. 101 _roll 101 _pitch 101 _yaw 101 ”, where t 101 101 is the ending moment of performing the action to be measured, and roll 101 _pitch 101 _yaw 101 101 _pitch 101 _yaw 101 is the attitude angle corresponding to the ending moment.

[0056] In an exemplary embodiment, after receiving the first action sequence, it is determined whether there is a target action sequence corresponding to the first action sequence in the database. If there is a target action sequence corresponding to the first action sequence in the database, the target motion trajectory corresponding to the target action sequence is determined as the first motion trajectory of the simulation component 121. The database herein may, for example, have pre-configured the mapping relationship between the action sequence and the motion trajectory of the simulation component 121. After determining that there is a target action sequence corresponding to the first action sequence in the database, according to the above mapping relationship, the first motion trajectory of the simulation component 121 can be determined.

[0057] It should be noted that when the delay test method is performed by the controller 122 of the delay detection device 120, the database can be stored in the controller 122. When the delay test method is performed by the server 130, the database can be stored in the server 130.

[0058] In an exemplary embodiment, if there is no target action sequence corresponding to the first action sequence in the database, the pose information of the reference point at different time points during the user's performance of the action corresponding to the first action sequence is obtained. For example, after determining that there is no target action sequence corresponding to the first action sequence in the database, a request is sent to the display device 110 to obtain the pose information of the user's head at different time points during the user's performance of the action corresponding to the first action sequence. Then, the display device 110 uses its own positioning and tracking system to obtain the pose information of the user's head at different time points according to a preset sampling time interval. After receiving the pose information of the user's head at different time points sent by the display device 110, the pose information of the reference point at different time points can be determined according to the pose information of the user's head at different time points.

[0059] In other embodiments, if there is no target action sequence corresponding to the first action sequence in the database, the pose information of multiple points to be measured at different time points during the user's performance of the action corresponding to the first action sequence is obtained; then, the pose information of the reference point at different time points is determined according to the pose information of the multiple points to be measured at different time points, where the reference point is the center point of the three-dimensional space formed by the multiple points to be measured. Specifically, after determining that there is no target action sequence corresponding to the first action sequence in the database, a request is sent to a motion capture system (not shown) to obtain the pose information of multiple points to be measured at different time points during the user's performance of the action according to the action to be measured. From the start time to the end time of the user's performance of the action to be measured, the motion capture system can obtain the pose information of multiple points to be measured at different time points according to a preset sampling time interval. For example, as Figure 5As shown, the number of measurement points to be measured can be six. The six measurement points to be measured can include a measurement point A on the user's left ear, a measurement point B on the user's left forehead, a measurement point C under the user's left eye, a measurement point D on the user's right ear, a measurement point E on the user's right forehead, and a measurement point F under the user's right eye. The center point O of the three-dimensional space formed by the above six measurement points to be measured can be the reference point. After receiving the pose information of multiple measurement points to be measured at different time points sent by the motion capture system, the pose information of the reference point (such as the center point O) at different time points can be determined according to the pose information of multiple measurement points to be measured at different time points.

[0060] After determining the pose information of the reference point at different time points, the first motion trajectory of the simulation component 121 in the preset time period can be determined based on the pose information of the reference point at different time points. For example, the pose information of the simulation component 121 at different time points is determined according to the pose information of the reference point at different time points. Then, the motion parameters of the simulation component 121 at different time points are determined according to the pose information of the simulation component 121 at different time points. Among them, the motion parameters of the simulation component 121 can include, but are not limited to, one or more of the rotation angle, speed, and acceleration of the simulation component 121. The rotation angle of the simulation component 121 can be determined according to the pose information of the simulation component 121 at two adjacent time points. The speed and acceleration of the simulation component 121 can be determined according to the rotation angle of the simulation component 121, the time interval between two adjacent time points, and the action execution time. Finally, the third motion trajectory is generated based on the pose information and motion parameters of the simulation component 121 at different time points, and the third motion trajectory is determined as the first motion trajectory of the simulation component 121 in the preset time period.

[0061] In this embodiment, the motion sequence of the user's head is collected, and the motion sequence of the user's head is mapped to the simulation component through a motion planning and control algorithm. The simulation component can be, for example, a simulation head mold, which meets the needs of personalized testing and makes the actions of the simulation component more realistic.

[0062] In an exemplary embodiment, the pose of the simulation component 121 can be controlled by three driving motors. The three driving motors can include a first driving motor, a second driving motor, and a third driving motor. The first driving motor can be used to control the roll angle of the simulation component 121, the second driving motor can be used to control the pitch angle of the simulation component 121, and the third driving motor can be used to control the yaw angle of the simulation component 121. Accordingly, the rotation angle of the simulation component 121 can include the rotation angles of the roll angle, pitch angle, and yaw angle of the simulation component 121.

[0063] In an exemplary embodiment, when generating a third motion trajectory based on the pose information and motion parameters of the simulation component 121 at different time points, it is determined whether the actual rotation angle of the simulation component 121 corresponds to the target rotation angle. If the actual rotation angle of the simulation component 121 does not correspond to the target rotation angle, the control signal of the drive motor is adjusted so that the actual rotation angle of the simulation component 121 corresponds to the target rotation angle, thereby making the third motion trajectory smoother.

[0064] In an exemplary embodiment, when the motion of the drive motor (e.g., the first drive motor, the second drive motor, the third drive motor) exceeds the safe range, an alarm is given and the drive motor is controlled to automatically stop moving. At the same time, software limits or hardware limits can be added to prevent the motion of the drive motor from exceeding the safe range and improve safety.

[0065] In this embodiment, by obtaining the first motion trajectory of the simulation component 121 in a preset time period through the first action sequence, the actions of different users can be simulated by the simulation component 121, and the actions of the simulation component 121 can be made more realistic, which is beneficial for personalized testing for different users, provides a wider range of test data for R & D personnel, better improves the display device, and reduces the delay of the display device. At the same time, the third motion trajectory generated according to the pose information and motion parameters of the simulation component 121 is smoother and more realistic.

[0066] In step S220, a second action sequence is obtained according to the display image of the display device during the movement of the simulation component according to the first motion trajectory, and a second motion trajectory in a preset time period is determined according to the second action sequence, where the second action sequence includes second time information and second pose information.

[0067] Specifically, after obtaining the first motion trajectory of the simulation component 121 in a preset time period, the simulation component 121 can move according to the first motion trajectory. From the start time to the end time when the simulation component 121 moves according to the first motion trajectory, the display images of the display device 110 at different time points are obtained, where the display images correspond to the motion of the simulation component 121.

[0068] Since there are moiré patterns in the display image, these moiré patterns will affect the accuracy of the feature points extracted from the display image. Therefore, after obtaining the display image, the display image is subjected to filtering processing, such as Gaussian blur and other processing, so as to reduce the influence of the moiré patterns on the accuracy of the feature points extracted from the display image.

[0069] After performing processing such as Gaussian blur on the displayed image, feature extraction is performed on the displayed images of the display device 110 at different time points to obtain feature points, and the two-dimensional coordinates of the feature points at different time points are determined. The number of feature points can be, for example, multiple. Since each feature point has a unique identity identifier, and there is a predefined mapping relationship between the identity identifier of the feature point and the three-dimensional coordinates, therefore, according to the identity identifier of the feature point and the above mapping relationship, the three-dimensional coordinates of the feature point at different time points can be determined, and a two-dimensional coordinate - three-dimensional coordinate matching point pair of the feature point at different time points can be constructed. Then, based on the two-dimensional coordinate - three-dimensional coordinate matching point pairs of the feature points at different time points, a second action sequence can be determined. For example, pose calculation is performed on the two-dimensional coordinate - three-dimensional coordinate matching point pairs of the feature points at different time points to obtain the rotation matrix of the displayed images at different time points relative to the world coordinate system; then, based on the rotation matrix of the displayed images at different time points relative to the world coordinate system, the second action sequence is determined. Specifically, the PNP (Perspective-n-Points) algorithm can be used to obtain the rotation matrix of the displayed images at different time points relative to the world coordinate system. After determining the second action sequence, the second motion trajectory of the preset time period can be determined according to the second action sequence.

[0070] The action parameters within the second action sequence can be represented in the form of "t n _roll n _pitch n _yaw n ", where 1 ≤ n ≤ b, b is the number of action parameters within the second action sequence, t n is the time corresponding to the nth action parameter within the second action sequence, roll n _pitch n _yaw n is the pose angle corresponding to the nth action parameter within the second action sequence. The above t n is the second time information of the second action sequence, and the above roll n _pitch n _yaw n is the second pose information of the second action sequence. The number of action parameters within the second action sequence can be the same as the number of action parameters within the first action sequence. The sampling time interval between adjacent action parameters within the second action sequence can be the same as the sampling time interval between adjacent action parameters within the first action sequence.

[0071] In this embodiment, preprocessing such as Gaussian blur is performed on the displayed image, which can reduce moiré patterns and improve the accuracy of feature points extracted from the displayed image. At the same time, two-dimensional coordinate - three-dimensional coordinate matching point pairs of feature points at different time points are constructed, and the pose changes of the virtual world at different time points are calculated based on the two-dimensional coordinate - three-dimensional coordinate matching point pairs of feature points at different time points to obtain a second action sequence. Finally, a second motion trajectory for a preset time period is determined according to the second action sequence.

[0072] In step S230, the delay of the display device in the preset time period is determined based on the first motion trajectory and the second motion trajectory.

[0073] In the prior art, when testing the delay of the display device 110, usually only the delay at the start time and / or end time of the user's action can be obtained, and the delay at other times during the user's action except for the start time and end time cannot be obtained. However, the delay at other times during the user's action is also relatively important, and it still affects the user's immersion.

[0074] To implement the delay test of the display device 110 in the preset time period, the first motion trajectory and the second motion trajectory of the preset time can be obtained, and after obtaining the first motion trajectory and the second motion trajectory, linear fitting and / or quadratic curve fitting are performed on the first motion trajectory and the second motion trajectory. For example, the linear intervals and non-linear intervals of the first motion trajectory and the second motion trajectory are identified, linear fitting is performed on the linear intervals, and quadratic curve fitting is performed on the non-linear intervals.

[0075] Figure 6 FIG. is a schematic diagram of the first motion trajectory and the second motion trajectory according to an exemplary embodiment of the present disclosure. Figure 7 For Figure 6 FIG. is a schematic diagram of linear fitting of the first motion trajectory in region G in, and the formula for linear fitting of the first motion trajectory in region G is y1 = -6.198 * x1 + 3.098, where x1 is the time corresponding to the first motion trajectory and y1 is the attitude angle corresponding to the first motion trajectory. Figure 8 For Figure 6 FIG. is a schematic diagram of linear fitting of the second motion trajectory in region G in, and the formula for linear fitting of the second motion trajectory in region G is y2 = -6.21 * x2 + 3.043, where x2 is the time corresponding to the second motion trajectory and y2 is the attitude angle corresponding to the second motion trajectory. It should be understood that the formulas for linear fitting of the first motion trajectory and the second motion trajectory in region G are only exemplary, and the present disclosure does not make specific limitations thereto.

[0076] As Figure 6 、 Figure 7 AndFigure 8 As shown, linear fitting is performed on the first motion trajectory and the second motion trajectory in the region G respectively, and the fitted first motion trajectory and the second motion trajectory are obtained respectively. The fitted first motion trajectory and the second motion trajectory can be referred to Figure 9 , it should be noted that Figure 9 Shown are the fitted first motion trajectory and the second motion trajectory within 0.3 seconds to 0.7 seconds.

[0077] By performing linear fitting on the linear intervals of the first motion trajectory and the second motion trajectory and performing quadratic curve fitting on the non-linear intervals, the fitting effect of the first motion trajectory and the second motion trajectory can be improved, which is beneficial to improving the test accuracy of the delay of the display device. In other examples, before fitting the first motion trajectory and the second motion trajectory, zero-offset preprocessing can be performed on the first motion trajectory and the second motion trajectory.

[0078] After fitting the first motion trajectory and the second motion trajectory, the delay of the display device 110 in the preset time period can be determined according to the fitted first motion trajectory and the second motion trajectory. As Figure 6 shown, both the first motion trajectory and the second motion trajectory are time-pose curves, the abscissa of which is time and the ordinate is the sitting posture (for example, the attitude angle). Both the first motion trajectory and the second motion trajectory can represent the pose changes at different time points. The time difference between the second motion trajectory and the first motion trajectory at the same pose is the delay of the display device 110. Therefore, according to the time difference between the fitted second motion trajectory and the first motion trajectory at the same pose, the delay curve of the display device 110 can be obtained, and then based on the delay curve of the display device 110, the delay of the display device 110 in the preset time period can be determined. Through the fitted first motion trajectory and the second motion trajectory, the delay test of the display device in the preset time period can be realized, and the test accuracy of the delay of the display device can be improved.

[0079] In other exemplary embodiments, as Figure 6 , Figure 9 and Figure 10 shown, the time-pose curve can be a time-attitude angle curve. According to the time difference between the fitted second motion trajectory and the first motion trajectory at the same attitude angle, the delay curve of the display device 110 can be obtained, where the attitude angle can include the roll angle, the pitch angle, and the yaw angle.

[0080] The time - pose curve may include a time - roll angle curve, a time - pitch angle curve, and a time - yaw angle curve. Correspondingly, the first motion trajectory may include a time - roll angle curve I, a time - pitch angle curve I, and a time - yaw angle curve I, and the second motion trajectory may include a time - roll angle curve II, a time - pitch angle curve II, and a time - yaw angle curve II. The delay curve of the display device 110 may include a roll angle delay curve, a pitch angle delay curve, and a yaw angle delay curve.

[0081] The roll angle delay curve of the display device 110 can be obtained according to the difference in time between the time - roll angle curve II and the time - roll angle curve I after fitting at the same roll angle. The pitch angle delay curve of the display device 110 can be obtained according to the difference in time between the time - pitch angle curve II and the time - pitch angle curve I after fitting at the same pitch angle. The yaw angle delay curve of the display device 110 can be obtained according to the difference in time between the time - yaw angle curve II and the time - yaw angle curve I after fitting at the same yaw angle.

[0082] In this embodiment, by linearly fitting the linear intervals of the first motion trajectory and the second motion trajectory and performing quadratic curve fitting on the non - linear intervals, the fitting effect of the first motion trajectory and the second motion trajectory can be improved. Then, the delay test for the preset time period of the display device can be realized through the delay curve obtained according to the fitted first motion trajectory and second motion trajectory, and the test accuracy of the delay of the display device can be improved.

[0083] Figure 11 It is an interaction schematic diagram of a display device and a delay detection device according to an exemplary embodiment of the present disclosure. The following will be combined with Figure 11 to describe the specific test process of the delay of the display device.

[0084] S31, the display device 110 sends a first action sequence to the controller 122 of the delay detection device 120.

[0085] S32, after receiving the first action sequence, the controller 122 determines whether there is a target action sequence corresponding to the first action sequence in the database. In response to the database having a target action sequence corresponding to the first action sequence, the target motion trajectory corresponding to the target action sequence is determined as the first motion trajectory of the simulation component.

[0086] S33, in response to the database not having a target action sequence corresponding to the first action sequence, the controller 122 sends a request to the display device 110 to obtain the pose information of the user's head at different time points during the user's action corresponding to the first action sequence.

[0087] S34. After receiving the request for obtaining the pose information, the display device 110 sends the pose information of the user's head at different time points during the user's actions corresponding to the first action sequence to the controller 122.

[0088] S35. After receiving the above-mentioned request for obtaining the pose information, the controller 122 determines the pose information of the reference point at different time points according to the pose information of the user's head at different time points, and determines the first motion trajectory of the simulation component during the preset time period according to the pose information of the reference point at different time points.

[0089] S36. After determining the first motion trajectory according to step S32 or S35, the controller 122 sends a request to the camera module 124 of the delay detection device 120 to obtain the display image of the display device during the movement of the simulation component according to the first motion trajectory.

[0090] S37. After receiving the above-mentioned request for obtaining the display image, the camera module 124 sends the display image of the display device during the movement of the simulation component according to the first motion trajectory to the controller 122.

[0091] S38. After receiving the above-mentioned display image, the controller 122 obtains the second action sequence according to the display image, and determines the second motion trajectory during the preset time period according to the second action sequence.

[0092] S39. After determining the second motion trajectory, the controller 122 determines the delay of the display device during the preset time period based on the first motion trajectory and the second motion trajectory.

[0093] In other embodiments, the delay detection device 120 may have a collection module (not shown). The collection module can be used to obtain the first action sequence and send the first action sequence to the controller 122 of the delay detection device 120. The collection module can also be used to obtain the pose information of multiple points to be measured at different time points during the user's actions corresponding to the first action sequence, and send the pose information to the controller 122 of the delay detection device 120.

[0094] Figure 12 It is a schematic diagram of the interaction between the display device, the delay detection device, and the server 130 according to an exemplary embodiment of the present disclosure. The following will be combined with Figure 12 to describe the specific test process of the delay of the display device.

[0095] S41. The display device 110 sends the first action sequence to the server 130.

[0096] S42. After receiving the first action sequence, the server 130 determines whether a target action sequence corresponding to the first action sequence exists in the database. In response to the existence of a target action sequence corresponding to the first action sequence in the database, the target motion trajectory corresponding to the target action sequence is determined as the first motion trajectory of the simulation component.

[0097] S43. In response to the non-existence of a target action sequence corresponding to the first action sequence in the database, the server 130 sends a request to the display device 110 to obtain the pose information of the user's head at different time points during the user's performance of the action corresponding to the first action sequence.

[0098] S44. After receiving the above request for obtaining pose information, the display device 110 sends the pose information of the user's head at different time points during the user's performance of the action corresponding to the first action sequence to the server 130.

[0099] S45. After receiving the above request for obtaining pose information, the server 130 determines the pose information of the reference point at different time points based on the pose information of the user's head at different time points, and determines the first motion trajectory of the simulation component during the preset time period based on the pose information of the reference point at different time points.

[0100] S46. After determining the first motion trajectory according to step S42 or S45, the server 130 sends a request to the controller 122 of the delay detection device 120 to obtain the display image of the display device during the movement of the simulation component according to the first motion trajectory.

[0101] S47. After receiving the above request for obtaining the display image, the controller 122 sends the display image of the display device during the movement of the simulation component according to the first motion trajectory to the server 130.

[0102] S48. After receiving the above display image, the server 130 obtains the second action sequence according to the display image, and determines the second motion trajectory during the preset time period according to the second action sequence.

[0103] S49. After determining the second motion trajectory, the server 130 determines the delay of the display device during the preset time period based on the first motion trajectory and the second motion trajectory.

[0104] In other embodiments, the delay detection device 120 may have an acquisition module (not shown). The acquisition module can be used to obtain the first action sequence and send the first action sequence to the server 130. The acquisition module can also be used to obtain the pose information of multiple points to be measured at different time points during the user's performance of the action corresponding to the first action sequence, and send the pose information to the server 130.

[0105] An exemplary embodiment of the present disclosure further provides a delay detection device, which may include an analog component, a driving motor, a camera module, and a controller. The analog component may be used to simulate the actions of a user. The analog component may be, for example, a simulation head mold. The driving motor may be used to drive the analog component to move, for example, to drive the analog component to move along a first motion trajectory, where the first motion trajectory may be determined based on a first action sequence, and the first action sequence may include first time information and first pose information. The camera module may be used to collect images, for example, to collect the display images of the display device under test during the movement of the analog component along the first motion trajectory. The controller may be communicatively connected to the driving motor and the camera module, for example. Wherein, the controller may be configured to: determine a second motion trajectory of a preset time period based on the display image, and determine the delay of the display device under test during the preset time period based on the first motion trajectory and the second motion trajectory.

[0106] In other examples, the second motion trajectory may be determined based on a second action sequence obtained from the display image, and the second action sequence may include second time information and second pose information.

[0107] In other examples, the controller may obtain the first motion trajectory of the preset time period according to the first action sequence, and control the driving motor to drive the analog component to move along the first motion trajectory.

[0108] It should be noted that the controller of the delay detection device may execute the above-mentioned delay test method 200.

[0109] An exemplary embodiment of the present disclosure further provides an electronic device, which includes at least one processor and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned delay test method 200.

[0110] An exemplary embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned delay test method 200 is implemented.

[0111] Figure 13 A block diagram of an electronic device 500 suitable for implementing the exemplary embodiments of the present disclosure is schematically shown. As Figure 13As shown, the electronic device 500 includes a processor 501, which can execute various appropriate steps and processes according to computer program instructions stored in a read-only memory (ROM) 502 or computer program instructions loaded from a memory 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0112] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506; an output unit 507; a memory 508, such as a magnetic disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0113] The processor 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 501 can execute the various methods and processes described above, such as executing the above-mentioned delay test method 200. For example, in some embodiments, the above-mentioned delay test method 200 can be implemented as a computer software program, which is stored in a machine-readable medium, such as the memory 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the processor 501, the above-mentioned delay test method 200 can be executed. Alternatively, in other embodiments, the processor 501 can be configured to execute the above-mentioned delay test method 200 in any other appropriate manner (e.g., by means of firmware).

[0114] Aspects of the present disclosure have been described herein with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to exemplary embodiments of the present disclosure. It should be understood that each step of the flowcharts and / or block diagrams, and combinations of steps in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0115] These computer-readable program instructions can be provided to a processor, a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus's processing unit, thereby producing a machine such that, when the instructions are executed by the processing unit of the computer or other programmable data processing apparatus, a device is produced that implements the functions / steps specified in one or more of the steps of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, a programmable data processing apparatus, and / or other devices to operate in a particular manner, so that the computer-readable medium storing the instructions comprises a manufacture, which includes instructions that implement various aspects of the functions / steps specified in one or more of the steps of the flowchart and / or block diagram.

[0116] The computer-readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices, such that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other devices implement the functions / steps specified in one or more of the steps of the flowchart and / or block diagram.

[0117] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each step in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the steps may occur in a different order than noted in the figures. For example, two consecutive steps may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each step in the block diagram and / or flowchart, and combinations of steps in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0118] The foregoing description is only an exemplary embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of protection involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the technical concept. For example, technical solutions formed by mutually replacing the above features with other technical features (but not limited to) having similar functions disclosed in the present disclosure.

Claims

1. A delay test method, characterized in that: include: In response to receiving a first action sequence, obtaining a first motion trajectory of the simulation component in a preset time period according to the first action sequence, wherein the first action sequence includes first time information and first posture information; Acquire a second motion sequence according to a display image of a display device during the period when the simulation component moves according to the first motion trajectory, and determine a second motion trajectory of the preset time period according to the second motion sequence, wherein the second motion sequence includes second time information and second posture information, and the second motion trajectory and the first motion trajectory are time-posture curves; and Determining the delay of the display device in the preset time period based on the first motion trajectory and the second motion trajectory includes: performing linear fitting and / or quadratic curve fitting on the first motion trajectory and the second motion trajectory; obtaining the delay curve of the display device according to the time difference between the second motion trajectory and the first motion trajectory after fitting in the same posture; and determining the delay of the display device in the preset time period based on the delay curve of the display device.

2. The method according to claim 1, wherein: Acquiring a first motion trajectory of a simulation component in a preset time period according to the first action sequence includes: In response to the database having a target action sequence corresponding to the first action sequence, a target motion trajectory corresponding to the target action sequence is determined as the first motion trajectory.

3. The method according to claim 1, wherein: Acquiring a first motion trajectory of a simulation component in a preset time period according to the first action sequence includes: In response to the database not having a target action sequence corresponding to the first action sequence, acquiring position information of a reference point at different time points during a user performing an action corresponding to the first action sequence; and The first motion trajectory is determined based on the position and posture information of the reference point at different time points.

4. The method according to claim 3, wherein: Acquiring position information of the reference point at different time points during the user performing an action corresponding to the first action sequence, including: Acquire position information of a plurality of points to be measured at different time points during the time when the user performs an action corresponding to the first action sequence; and The position and posture information of the reference point at different time points is determined according to the position and posture information of the plurality of points to be measured at different time points, wherein the reference point is the center point of the three-dimensional space formed by the plurality of points to be measured.

5. The method according to claim 3, wherein: Determining the first motion trajectory based on the position information of the reference point at different time points includes: Determine the pose information of the simulation component at different time points according to the pose information of the reference point at different time points; Determining motion parameters of the simulation component at different time points according to the posture information of the simulation component at different time points, wherein the motion parameters include one or more of the rotation angle, speed and acceleration of the simulation component; and A third motion trajectory is generated based on the posture information and motion parameters of the simulation component at different time points, and the third motion trajectory is determined as the first motion trajectory.

6. The method according to any one of claims 1 to 5, wherein: Acquiring a second action sequence according to a display image of a display device during the period when the simulation component moves along the first motion trajectory, comprising: Extracting features from the displayed image to obtain feature points, and determining the two-dimensional coordinates of the feature points at different time points; Determine the three-dimensional coordinates of the feature points at different time points according to the identity identifiers of the feature points, and construct two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature points at different time points; and The second action sequence is determined based on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature points at different time points.

7. The method according to claim 6, wherein: Determining the second action sequence based on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature points at different time points includes: Performing pose calculation on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature points at different time points to obtain the rotation matrix of the display image at different time points relative to the world coordinate system; and The second action sequence is determined based on the rotation matrix at different time points.

8. The method according to claim 6, wherein: Before extracting features from the displayed image to obtain feature points, the method further includes: Gaussian blur processing is performed on the displayed image.

9. The method according to any one of claims 1 to 5, wherein: Performing linear fitting and / or quadratic curve fitting on the first motion trajectory and the second motion trajectory includes: identifying linear intervals and nonlinear intervals of the first motion trajectory and the second motion trajectory; and A linear fitting is performed on the linear interval, and a quadratic curve fitting is performed on the nonlinear interval.

10. The method according to any one of claims 1 to 5, wherein: The time-posture curve is a time-posture angle curve, and the delay curve of the display device is obtained according to the time difference between the second motion trajectory and the first motion trajectory after fitting under the same posture, including: The delay curve of the display device is obtained according to the time difference between the second motion trajectory and the first motion trajectory after fitting at the same attitude angle, wherein the attitude angle includes a roll angle, a yaw angle, and a pitch angle.

11. A time delay detection device, characterized in that: include: Simulation components, used to simulate user actions; A driving motor, configured to drive the simulation component to move along a first motion trajectory, wherein the first motion trajectory is determined based on a first action sequence including first time information and first posture information; a camera module, used for capturing a display image of the display device under test when the simulation component moves along the first motion trajectory; and A controller is connected to the driving motor and the camera module for communication. The controller is configured to: determine a second motion trajectory of a preset time period according to the display image, and determine the delay of the display device to be tested in the preset time period based on the first motion trajectory and the second motion trajectory, wherein the second motion trajectory is determined based on a second action sequence obtained from the display image, and the second action sequence includes second time information and second posture information, Among them, the second motion trajectory and the first motion trajectory are time-posture curves, and determining the delay of the display device in the preset time period based on the first motion trajectory and the second motion trajectory includes: performing linear fitting and / or quadratic curve fitting on the first motion trajectory and the second motion trajectory; obtaining the delay curve of the display device according to the time difference between the second motion trajectory and the first motion trajectory after fitting under the same posture; and determining the delay of the display device in the preset time period based on the delay curve of the display device.

12. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the delay testing method according to any one of claims 1 to 10.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the delay testing method according to any one of claims 1 to 10 is implemented.

Images