Delay test method, delay detection device, electronic device and storage medium

Abstract

The invention provides a time delay test method, time delay detection equipment, electronic equipment and a storage medium, and particularly relates to the related technical field of time delay test.The time delay test method comprises the steps that in response to a received first action sequence, a first movement track of a simulation assembly in a preset time period is obtained according to the first action sequence, the first action sequence comprises first time information and first pose information; a second action sequence is obtained according to a display image of the display device in the period when the simulation assembly moves according to the first movement track, a second movement track in the preset time period is determined according to the second action sequence, and the second action sequence comprises second time information and second pose information; and determining the time delay of the display equipment in the preset time period based on the first movement track and the second movement track.

Description

Technical Field

[0001] The present disclosure relates to the technical field related to delay testing, and in particular to a delay testing method, a delay detection device, an electronic device and a storage medium. Background Art

[0002] With the development of science and technology, higher requirements are placed on the performance of display devices such as virtual reality devices, augmented reality devices, and mixed reality devices. MTP (motion-to-photon) delay refers to the time from the start of user movement to the display of the corresponding image on the screen. It is an important parameter reflecting the performance of display devices. For users, the smaller the MTP delay, the better the user's immersion; the larger the MTP delay, the stronger the user's dizziness.

[0003] The MTP delay test is a process that needs to be carried out before the display device leaves the factory. The test accuracy of the MTP delay is relatively important, which can guide R&D personnel to better improve the display device, thereby reducing the MTP delay of the display device. However, the test accuracy of the MTP delay measured by the existing test method is low. Summary of the invention

[0004] A first aspect of the present disclosure provides a delay testing method, which includes: in response to receiving a first action sequence, obtaining a first motion trajectory of a simulation component in a preset time period according to the first action sequence, wherein the first action sequence includes first time information and first posture information; obtaining a second action sequence according to a display image of a display device during movement of the simulation component according to the first motion trajectory, and determining a second motion trajectory of a preset time period according to the second action sequence, wherein the second action sequence includes second time information and second posture information; and determining the delay of the display device in the preset time period based on the first motion trajectory and the second motion trajectory.

[0005] In some embodiments, obtaining a first motion trajectory of a simulation component in a preset time period according to a first motion sequence includes: in response to a target motion sequence corresponding to the first motion sequence existing in a database, determining a target motion trajectory corresponding to the target motion sequence as the first motion trajectory.

[0006] In some embodiments, obtaining a first motion trajectory of a simulation component in a preset time period according to a first action sequence includes: in response to a database not having a target action sequence corresponding to the first action sequence, obtaining position information of a reference point at different time points during a user's action corresponding to the first action sequence; and determining the first motion trajectory based on the position information of the reference point at different time points.

[0007] In some embodiments, obtaining the posture information of the reference point at different time points during the user's action corresponding to the first action sequence includes: obtaining the posture information of multiple points to be measured at different time points during the user's action corresponding to the first action sequence; and determining the posture information of the reference point at different time points based on the posture information of the multiple points to be measured at different time points, wherein the reference point is the center point of the three-dimensional space formed by the multiple points to be measured.

[0008] In some embodiments, determining a first motion trajectory based on posture information of a reference point at different time points includes: determining the posture information of a simulation component at different time points based on the posture information of the reference point at different time points; determining motion parameters of the simulation component at different time points based on the posture information of the simulation component at different time points, wherein the motion parameters include one or more of a rotation angle, speed, and acceleration of the simulation component; and generating a third motion trajectory based on the posture information and motion parameters of the simulation component at different time points, and determining the third motion trajectory as the first motion trajectory.

[0009] In some embodiments, acquiring a second action sequence based on a display image of a display device during movement of a simulated component along a first motion trajectory includes: performing feature extraction on the display image to acquire feature points, and determining the two-dimensional coordinates of the feature points at different time points; determining the three-dimensional coordinates of the feature points at different time points based on identity identifiers of the feature points, and constructing two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature points at different time points; and determining the second action sequence based on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature points at different time points.

[0010] In some embodiments, determining the second action sequence based on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of feature points at different time points includes: performing pose calculations on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of feature points at different time points, and obtaining the rotation matrix of the displayed image at different time points relative to the world coordinate system; and determining the second action sequence based on the rotation matrix at different time points.

[0011] In some implementations, before performing feature extraction on the displayed image to obtain feature points, the method further includes: performing Gaussian blur processing on the displayed image.

[0012] In some embodiments, determining the delay of the display device in a preset time period based on the first motion trajectory and the second motion trajectory includes: performing linear fitting and / or quadratic curve fitting on the first motion trajectory and the second motion trajectory; and determining the delay of the display device in the preset time period based on the fitted first motion trajectory and the second motion trajectory.

[0013] In some embodiments, performing linear fitting and / or quadratic curve fitting on the first motion trajectory and the second motion trajectory includes: identifying linear intervals and nonlinear intervals of the first motion trajectory and the second motion trajectory; and performing linear fitting on the linear intervals and performing quadratic curve fitting on the nonlinear intervals.

[0014] In some embodiments, the first motion trajectory and the second motion trajectory are time-position curves. Determining the delay of the display device in a preset time period according to the fitted first motion trajectory and the second motion trajectory includes: obtaining the delay curve of the display device according to the time difference between the fitted second motion trajectory and the first motion trajectory in the same position; and determining the delay of the display device in the preset time period based on the delay curve of the display device.

[0015] In some embodiments, the time-position curve is a time-position angle curve. Acquiring the delay curve of the display device according to the time difference between the second motion trajectory and the first motion trajectory after fitting under the same position includes: acquiring the delay curve of the display device according to the time difference between the second motion trajectory and the first motion trajectory after fitting under the same attitude angle, wherein the attitude angle includes a roll angle, a yaw angle, and a pitch angle.

[0016] A second aspect of the present disclosure provides a delay detection device, which includes a simulation component, a drive motor, a camera module and a controller. The simulation component is used to simulate the user's actions. The drive motor is used to drive the simulation component to move along a first motion trajectory, wherein the first motion trajectory is determined based on a first action sequence including first time information and first posture information. The camera module is used to capture a display image of the display device to be tested during the period when the simulation component moves along the first motion trajectory. The controller is communicatively connected to the drive motor and the camera module, wherein the controller is configured to: determine a second motion trajectory of a preset time period based on the display image, and determine the delay of the display device to be tested in the preset time period based on the first motion trajectory and the second motion trajectory.

[0017] The third aspect of the present disclosure provides an electronic device, which includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the delay testing method of the first aspect of the present disclosure.

[0018] A fourth aspect of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the delay testing method of the first aspect of the present disclosure is implemented.

[0019] The present invention obtains a first motion trajectory of a simulation component in a preset time period through a first action sequence, and can use the simulation component to simulate the actions of different users and make the actions of the simulation component more realistic; then, while the simulation component moves according to the first motion trajectory, a second motion trajectory of the preset time period is determined according to a second action sequence obtained from a display image of the display device, and the delay of the display device in the preset time period is determined according to the first motion trajectory and the second motion trajectory, thereby realizing a delay test of the preset time period and improving the test accuracy of the delay of the display device.

[0020] It should be understood that the content described in this section is not intended to identify the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Other features, objectives and advantages of the disclosed embodiments will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings, in which: Figure 1 An exemplary system framework applicable to the delay testing method according to the present disclosure is shown; Figure 2 Another exemplary system framework applicable to the delay testing method according to the present disclosure is shown; Figure 3 A schematic flow chart of a delay test method according to an exemplary embodiment of the present disclosure is shown; Figure 4 A schematic diagram showing an attitude angle according to an exemplary embodiment of the present disclosure; Figure 5 A schematic diagram showing the distribution of points to be tested according to an exemplary embodiment of the present disclosure is shown; Figure 6 A schematic diagram showing a first motion trajectory and a second motion trajectory according to an exemplary embodiment of the present disclosure; Figure 7 Shows the Figure 6 Schematic diagram of linear fitting of the first motion trajectory in the region G; Figure 8 Shows the Figure 6 Schematic diagram of linear fitting of the second motion trajectory in the region G in FIG. Fig. 9 A schematic diagram showing a first motion trajectory and a second motion trajectory after fitting according to an exemplary embodiment of the present disclosure is shown; Fig.10 A schematic diagram showing a delay curve of a display device according to an exemplary embodiment of the present disclosure; Fig.11 A schematic diagram showing the interaction between a display device and a delay detection device according to an exemplary embodiment of the present disclosure is shown; Fig.12 A schematic diagram showing the interaction between a display device, a delay detection device and a server according to an exemplary embodiment of the present disclosure is shown; Fig.13 A schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0022] In order to better understand the present disclosure, various aspects of the present disclosure will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present disclosure and are not intended to limit the scope of the present disclosure in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0023] It should be noted that in this specification, the expressions first, second, etc. are only used to distinguish one feature from another feature, and do not represent any limitation on the feature. Therefore, without departing from the teaching of the present disclosure, the first motion trajectory discussed below may also be referred to as the second motion trajectory, and the second motion trajectory may also be referred to as the first motion trajectory.

[0024] It should also be understood that expressions such as "include", "comprising", "having", "including" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, the word "exemplarily" is intended to refer to an example or illustration.

[0025] Unless otherwise defined, all terms (including technical and scientific terms) used in the present disclosure have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs. It should also be understood that terms (such as those defined in commonly used dictionaries) should be understood to have the same meaning as they have in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense, unless explicitly defined in the present disclosure.

[0026] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other. In addition, unless explicitly limited or contradictory to the context, the specific steps included in the method described in the present disclosure are not necessarily limited to the order described, but can be performed in any order or in parallel. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] Figure 1 and Figure 2 An exemplary system architecture 100 of a delay testing method applicable to some embodiments of the present disclosure is respectively shown.

[0028] In an exemplary embodiment, Figure 1 As shown, the system architecture 100 may include a display device 110 and a delay detection device 120. The display device 110 may include one or more of a virtual reality device, an augmented reality device, and a mixed reality device. The delay detection device 120 may include a simulation component 121, a controller 122, a drive motor 123, and a camera module 124. The simulation component 121 may be used to simulate the user's actions. The drive motor 123 may be used to drive the simulation component 121 to move. The camera module 124 may be used to capture the display image of the display device 110 during the movement of the simulation component 121. The controller 122 may be connected to the drive motor 123 and the camera module 124, for example, by communication. The controller 122 may be connected to the display device 110, for example, by communication.

[0029] It should be noted that the delay testing method according to the exemplary embodiment of the present disclosure may be executed by the controller 122 of the delay detection device 120 .

[0030] In an exemplary embodiment, Figure 2 As shown, the system architecture 100 may include a display device 110, a delay detection device 120 and a server 130. The display device 110 may include one or more of a virtual reality device, an augmented reality device and a mixed reality device. The delay detection device 120 may include a simulation component 121, a controller 122, a drive motor 123 and a camera module 124. The simulation component 121 may be used to simulate the user's action. The drive motor 123 may be used to drive the simulation component 121 to move. The camera module 124 may be used to collect the display image of the display device 110 during the movement of the simulation component 121. The controller 122 may be connected to the drive motor 123 and the camera module 124, for example, by communication. The server 130 may be connected to the display device 110 and the controller 122, for example, by communication. The server 130 may provide various services based on the display device 110 and the delay detection device 120. For example, the server 130 may analyze and process the received first action sequence and generate a processing result (for example, obtaining the first motion trajectory of the simulation component 121 according to the first action sequence).

[0031] It should be noted that the server 130 can be hardware or software. When the server 130 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 130 is software, it can be implemented as multiple software or software modules (for example, used to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here. In addition, the delay test method of the exemplary embodiment of the present disclosure can be executed by the server 130.

[0032] Figure 3 2 is a flow chart of a delay test method 200 according to an exemplary embodiment of the present disclosure. The delay test method 200 may be performed by, for example, a controller 122 (such as Figure 1 ) or server 130 (such as Figure 2 ) and other electronic devices. It should be understood that the delay test method 200 may also include additional steps not shown and / or may omit the steps shown, and the scope of the present disclosure is not limited in this respect. The delay test method 200 may be applicable to application scenarios such as delay testing of a display device 110, wherein the display device 110 may include a virtual reality device, an augmented reality device, or a mixed reality device. It should be understood that the delay test method 200 may also be applied to application scenarios of delay testing of other display devices, which is not limited here.

[0033] like Figure 3 As shown, the delay test method 200 may include the following steps: S210. In response to receiving a first action sequence, obtaining a first motion trajectory of a simulation component in a preset time period according to the first action sequence, wherein the first action sequence includes first time information and first posture information.

[0034] S220, acquiring a second action sequence according to a display image of the display device during the period when the simulation component moves along the first motion trajectory, and determining a second motion trajectory of a preset time period according to the second action sequence, wherein the second action sequence includes second time information and second posture information.

[0035] S230: Determine a delay of the display device in a preset time period based on the first motion trajectory and the second motion trajectory.

[0036] The method provided by the exemplary embodiment of the present disclosure obtains a first motion trajectory of a simulation component in a preset time period through a first action sequence, and can use the simulation component to simulate the actions of different users and make the actions of the simulation component more realistic; then, while the simulation component moves according to the first motion trajectory, a second motion trajectory of the preset time period is determined according to a second action sequence obtained from a display image of the display device, and the delay of the display device in the preset time period can be determined according to the first motion trajectory and the second motion trajectory, thereby realizing a delay test of the preset time period and improving the test accuracy of the delay of the display device.

[0037] Steps S210 to S230 of the exemplary embodiment of the present disclosure are described in detail below.

[0038] In step S210, in response to receiving the first action sequence, a first motion trajectory of the simulation component in a preset time period is obtained according to the first action sequence, wherein the first action sequence may include first time information and first posture information.

[0039] In an exemplary embodiment, the first action sequence may be an action sequence corresponding to the action to be tested. For example, the first action sequence may be an action sequence constructed by the action parameters of different time points during which the user performs the action to be tested. The number of action parameters in the first action sequence may be determined by the action execution time and sampling time interval of the action to be tested. The sampling time intervals between two adjacent action parameters in the first action sequence may be the same or different. For example, the action execution time of the action to be tested is 1s, the sampling time interval is 0.01s, and the total number of samples from the start time to the end time of the execution of the action to be tested is 101, that is, the number of action parameters in the first action sequence is 101.

[0040] The action parameters in the first action sequence can be accessed through the "t m _roll m _pitch m _yaw m ", where 1≤m≤a, a is the number of action parameters in the first action sequence, t m is the time corresponding to the mth action parameter in the first action sequence, roll m _pitch m _yaw m is the attitude angle corresponding to the mth action parameter in the first action sequence. Figure 4 As shown in the figure, in the world coordinate system, roll is the angle of rotation around the Z axis (i.e., the roll angle), pitch is the angle of rotation around the X axis (i.e., the pitch angle), and yaw is the angle of rotation around the Y axis (i.e., the yaw angle). mis the first time information of the first action sequence, the above roll m _pitch m _yaw m is the first pose information of the first action sequence.

[0041] For example, the number of action parameters in the first action sequence may be 101. The first action parameter in the first action sequence may be "t 1 _roll 1 _pitch 1 _yaw 1 ”,t 1 is the starting time of executing the action to be tested, roll 1 _pitch 1 _yaw 1 is the attitude angle corresponding to the starting moment. 1 _roll 1 _pitch 1 _yaw 1 ” may be, for example, “000_10_20_30”, where “000” indicates the starting time, and “10_20_30” indicates that the roll angle at the starting time is 10°, the pitch angle is 20°, and the yaw angle is 30°. The 101st action parameter in the first action sequence may be “t 101 _roll 101 _pitch 101 _yaw 101 ”,t 101 Roll is the end time of executing the action to be tested. 101 _pitch 101 _yaw 101 is the attitude angle corresponding to the termination moment.

[0042] In an exemplary embodiment, after receiving the first action sequence, it is determined whether the database has a target action sequence corresponding to the first action sequence. If the database has a target action sequence corresponding to the first action sequence, the target motion trajectory corresponding to the target action sequence is determined as the first motion trajectory of the simulation component 121. The database herein may, for example, be a mapping relationship between a pre-configured action sequence and a motion trajectory of the simulation component 121. After determining that the database has a target action sequence corresponding to the first action sequence, the first motion trajectory of the simulation component 121 may be determined according to the above mapping relationship.

[0043] It should be noted that when the delay test method is implemented by the controller 122 of the delay detection device 120, the database may be stored in the controller 122. When the delay test method is implemented by the server 130, the database may be stored in the server 130.

[0044] In an exemplary embodiment, if the database does not contain the target action sequence corresponding to the first action sequence, the position information of the reference point at different time points during the user's action corresponding to the first action sequence is obtained. For example, after determining that the database does not contain the target action sequence corresponding to the first action sequence, a request is sent to the display device 110 to obtain the position information of the user's head at different time points during the user's action corresponding to the first action sequence, and then the display device 110 uses its own positioning tracking system to obtain the position information of the user's head at different time points according to a preset sampling time interval. After receiving the position information of the user's head at different time points sent by the display device 110, the position information of the reference point at different time points can be determined based on the position information of the user's head at different time points.

[0045] In other embodiments, if the database does not contain the target action sequence corresponding to the first action sequence, then the posture information of multiple points to be measured at different time points during the user's action corresponding to the first action sequence is obtained; then, the posture information of the reference point at different time points is determined based on the posture information of the multiple points to be measured at different time points, wherein the reference point is the center point of the three-dimensional space formed by the multiple points to be measured. Specifically, after determining that the database does not contain the target action sequence corresponding to the first action sequence, a request is sent to the motion capture system (not shown) to obtain the posture information of multiple points to be measured at different time points during the user's action according to the action to be measured. From the start time to the end time of the user's action according to the action to be measured, the motion capture system can obtain the posture information of multiple points to be measured at different time points according to a preset sampling time interval. For example, Figure 5 As shown, the number of points to be measured may be six, and the six points to be measured may include point A located at the user's left ear, point B located at the user's left forehead, point C located below the user's left eye, point D located at the user's right ear, point E located at the user's right forehead, and point F located below the user's right eye. The center point O of the three-dimensional space formed by the above six points to be measured may be the reference point. After receiving the position and posture information of the multiple points to be measured at different time points sent by the motion capture system, the position and posture information of the reference point (e.g., center point O) at different time points may be determined according to the position and posture information of the multiple points to be measured at different time points.

[0046] After determining the position information of the reference point at different time points, the first motion trajectory of the simulation component 121 in the preset time period can be determined based on the position information of the reference point at different time points. For example, the position information of the simulation component 121 at different time points is determined based on the position information of the reference point at different time points. Then, the action parameters of the simulation component 121 at different time points are determined based on the position information of the simulation component 121 at different time points, wherein the action parameters of the simulation component 121 may include but are not limited to one or more of the rotation angle, speed and acceleration of the simulation component 121. The rotation angle of the simulation component 121 can be determined based on the position information of the simulation component 121 at two adjacent time points. The speed and acceleration of the simulation component 121 can be determined based on the rotation angle of the simulation component 121, the time interval between two adjacent time points, and the action execution time. Finally, a third motion trajectory is generated based on the position information and action parameters of the simulation component 121 at different time points, and the third motion trajectory is determined as the first motion trajectory of the simulation component 121 in the preset time period.

[0047] In this embodiment, the movement sequence of the user's head is collected and mapped to the simulation component through motion planning and control algorithms. The simulation component can be, for example, a simulated head model to meet the needs of personalized testing and make the movement of the simulation component more realistic.

[0048] In an exemplary embodiment, the position and posture of the simulation component 121 can be controlled by three drive motors, which may include a first drive motor, a second drive motor, and a third drive motor. The first drive motor can be used to control the roll angle of the simulation component 121, the second drive motor can be used to control the pitch angle of the simulation component 121, and the third drive motor can be used to control the yaw angle of the simulation component 121. Accordingly, the rotation angle of the simulation component 121 may include the rotation angles of the roll angle, the pitch angle, and the yaw angle of the simulation component 121.

[0049] In an exemplary embodiment, when the third motion trajectory is generated based on the posture information and motion parameters of the simulation component 121 at different time points, it is determined whether the actual rotation angle of the simulation component 121 corresponds to the target rotation angle. If the actual rotation angle of the simulation component 121 does not correspond to the target rotation angle, the control signal of the drive motor is adjusted so that the actual rotation angle of the simulation component 121 corresponds to the target rotation angle, thereby making the third motion trajectory smoother.

[0050] In an exemplary embodiment, when the movement of the drive motor (e.g., the first drive motor, the second drive motor, and the third drive motor) exceeds the safety range, an alarm is sounded and the drive motor is controlled to automatically stop moving. At the same time, software limits or hardware limits can be added to prevent the movement of the drive motor from exceeding the safety range, thereby improving safety.

[0051] In this embodiment, the first motion trajectory of the simulation component 121 in a preset time period is obtained through the first action sequence, and the simulation component 121 can be used to simulate the actions of different users, and the actions of the simulation component 121 are made more realistic, which is conducive to personalized testing for different users, providing more extensive test data for R&D personnel, better improving the display device, and reducing the delay of the display device. At the same time, the third motion trajectory generated according to the posture information and action parameters of the simulation component 121 is smoother and more realistic.

[0052] In step S220, a second action sequence is acquired based on the display image of the display device during the simulation component moving along the first motion trajectory, and a second motion trajectory of a preset time period is determined based on the second action sequence, wherein the second action sequence includes second time information and second posture information.

[0053] Specifically, after obtaining the first motion trajectory of the simulation component 121 in the preset time period, the simulation component 121 can move according to the first motion trajectory. From the start time to the end time of the movement of the simulation component 121 according to the first motion trajectory, the display image of the display device 110 at different time points is obtained, wherein the display image corresponds to the movement of the simulation component 121.

[0054] Since moiré patterns exist in the displayed image, these moiré patterns will affect the accuracy of feature points extracted from the displayed image. Therefore, after the displayed image is acquired, the displayed image is filtered, such as by Gaussian blur, to reduce the influence of the moiré patterns on the accuracy of feature points extracted from the displayed image.

[0055] After the display image is subjected to Gaussian blur and other processing, feature extraction is performed on the display image of the display device 110 at different time points to obtain feature points, and the two-dimensional coordinates of the feature points at different time points are determined. The number of feature points may be, for example, multiple. Since each feature point has a unique identity, and there is a predefined mapping relationship between the identity of the feature point and the three-dimensional coordinates, the three-dimensional coordinates of the feature point at different time points can be determined according to the identity of the feature point and the above mapping relationship, and the two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature point at different time points can be constructed. Then, based on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature point at different time points, the second action sequence can be determined, for example, the two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature point at different time points are calculated, and the rotation matrix of the display image at different time points relative to the world coordinate system is obtained; then, the second action sequence is determined based on the rotation matrix of the display image at different time points relative to the world coordinate system. Specifically, the rotation matrix of the display image at different time points relative to the world coordinate system can be obtained by the PNP (Perspective-n-Points) algorithm. After the second action sequence is determined, the second motion trajectory of the preset time period can be determined according to the second action sequence.

[0056] The action parameters in the second action sequence can be passed through "t n _roll n _pitch n _yaw n ", where 1≤n≤b, b is the number of action parameters in the second action sequence, t n is the time corresponding to the nth action parameter in the second action sequence, roll n _pitch n _yaw n is the attitude angle corresponding to the nth action parameter in the second action sequence. n is the second time information of the second action sequence, the above roll n _pitch n _yaw n is the second posture information of the second motion sequence. The number of motion parameters in the second motion sequence may be the same as the number of motion parameters in the first motion sequence. The sampling time interval between adjacent motion parameters in the second motion sequence may be the same as the sampling time interval between adjacent motion parameters in the first motion sequence.

[0057] In this embodiment, preprocessing such as Gaussian blurring is performed on the displayed image to reduce moiré patterns and improve the accuracy of feature points extracted from the displayed image. At the same time, two-dimensional coordinate-three-dimensional coordinate matching point pairs of feature points at different time points are constructed, and the position and posture changes of the virtual world at different time points are calculated based on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of feature points at different time points, and the second action sequence is obtained, and finally the second motion trajectory of the preset time period is determined based on the second action sequence.

[0058] In step S230, a delay of the display device in a preset time period is determined based on the first motion trajectory and the second motion trajectory.

[0059] In the prior art, when testing the delay of the display device 110, usually only the delay of the start time and / or the end time of the user's action can be obtained, and the delay of other moments during the user's action except the start time and the end time cannot be obtained. However, the delay of other moments during the user's action is also relatively important, which will still affect the user's immersion.

[0060] In order to implement a delay test of a preset time period of the display device 110, a first motion trajectory and a second motion trajectory of the preset time can be obtained, and after obtaining the first motion trajectory and the second motion trajectory, linear fitting and / or quadratic curve fitting are performed on the first motion trajectory and the second motion trajectory. For example, the linear interval and the nonlinear interval of the first motion trajectory and the second motion trajectory are identified, and linear fitting is performed on the linear interval, and quadratic curve fitting is performed on the nonlinear interval.

[0061] Figure 6 Schematic diagram of a first motion trajectory and a second motion trajectory according to an exemplary embodiment of the present disclosure. Figure 7 For Figure 6 Schematic diagram of linear fitting of the first motion trajectory in area G, the formula for linear fitting of the first motion trajectory in area G is y1=-6.198*x1+3.098, where x1 is the time corresponding to the first motion trajectory, and y1 is the attitude angle corresponding to the first motion trajectory. Figure 8 For Figure 6 Schematic diagram of linear fitting of the second motion trajectory in region G in FIG. The formula for linear fitting of the second motion trajectory in region G is y2=-6.21*x2+3.043, wherein x2 is the time corresponding to the second motion trajectory, and y2 is the attitude angle corresponding to the second motion trajectory. It should be understood that the formula for linear fitting of the first motion trajectory and the second motion trajectory in region G is only exemplary, and the present disclosure does not make any specific limitation thereto.

[0062] like Figure 6 , Figure 7 and Figure 8 As shown, linear fitting is performed on the first motion trajectory and the second motion trajectory in the region G, and the fitted first motion trajectory and the second motion trajectory are obtained. The fitted first motion trajectory and the second motion trajectory can be referred to Fig. 9 It should be noted that Fig. 9 Shown are the fitted first motion trajectory and the second motion trajectory within 0.3 seconds to 0.7 seconds.

[0063] By performing linear fitting on the linear interval of the first motion trajectory and the second motion trajectory and performing quadratic curve fitting on the nonlinear interval, the fitting effect of the first motion trajectory and the second motion trajectory can be improved, which is beneficial to improving the delay test accuracy of the display device. In other examples, before fitting the first motion trajectory and the second motion trajectory, the first motion trajectory and the second motion trajectory can be preprocessed with zero bias.

[0064] After fitting the first motion trajectory and the second motion trajectory, the delay of the display device 110 in a preset time period can be determined according to the fitted first motion trajectory and the second motion trajectory. Figure 6 As shown, both the first motion trajectory and the second motion trajectory are time-posture curves, with the horizontal axis being time and the vertical axis being sitting posture (e.g., posture angle). Both the first motion trajectory and the second motion trajectory can represent posture changes at different time points, and the time difference between the second motion trajectory and the first motion trajectory under the same posture is the delay of the display device 110. Therefore, the delay curve of the display device 110 can be obtained according to the time difference between the second motion trajectory and the first motion trajectory after fitting under the same posture, and then the delay of the display device 110 in a preset time period can be determined based on the delay curve of the display device 110. The delay test of the preset time period of the display device can be implemented by the fitted first motion trajectory and the second motion trajectory, and the test accuracy of the delay of the display device can be improved.

[0065] In other exemplary embodiments, Figure 6 , Fig. 9 and Fig.10 As shown, the time-posture curve may be a time-attitude angle curve, and the delay curve of the display device 110 may be obtained according to the time difference between the second motion trajectory and the first motion trajectory fitted at the same attitude angle, wherein the attitude angle may include a roll angle, a pitch angle and a yaw angle.

[0066] The time-posture curve may include a time-roll angle curve, a time-pitch angle curve, and a time-yaw angle curve. Accordingly, the first motion trajectory may include a time-roll angle curve I, a time-pitch angle curve I, and a time-yaw angle curve I, and the second motion trajectory may include a time-roll angle curve II, a time-pitch angle curve II, and a time-yaw angle curve II. The delay curve of the display device 110 may include a roll angle delay curve, a pitch angle delay curve, and a yaw angle delay curve.

[0067] The roll angle delay curve of the display device 110 can be obtained according to the time difference between the time-roll angle curve II and the time-roll angle curve I after fitting at the same roll angle. The pitch angle delay curve of the display device 110 can be obtained according to the time difference between the time-pitch angle curve II and the time-pitch angle curve I after fitting at the same pitch angle. The yaw angle delay curve of the display device 110 can be obtained according to the time difference between the time-yaw angle curve II and the time-yaw angle curve I after fitting at the same yaw angle.

[0068] In this embodiment, by performing linear fitting on the linear interval of the first motion trajectory and the second motion trajectory and performing quadratic curve fitting on the nonlinear interval, the fitting effect of the first motion trajectory and the second motion trajectory can be improved, and then the delay test of the preset time period of the display device can be implemented by obtaining the delay curve according to the fitted first motion trajectory and the second motion trajectory, and the delay test accuracy of the display device can be improved.

[0069] Fig.11 FIG. 1 is a schematic diagram of the interaction between a display device and a delay detection device according to an exemplary embodiment of the present disclosure. Fig.11 To describe the specific testing process of the delay of the display device.

[0070] S31 , the display device 110 sends a first action sequence to the controller 122 of the delay detection device 120 .

[0071] S32, after receiving the first action sequence, the controller 122 determines whether there is a target action sequence corresponding to the first action sequence in the database. In response to the database having a target action sequence corresponding to the first action sequence, a target motion trajectory corresponding to the target action sequence is determined as the first motion trajectory of the simulation component.

[0072] S33, in response to the database not having a target action sequence corresponding to the first action sequence, the controller 122 sends a request to the display device 110 to obtain the posture information of the user's head at different time points during the user's action corresponding to the first action sequence.

[0073] S34, after receiving the request for obtaining the posture information, the display device 110 sends the posture information of the user's head at different time points during the user's action corresponding to the first action sequence to the controller 122.

[0074] S35, after receiving the above-mentioned acquired posture information, the controller 122 determines the posture information of the reference point at different time points according to the posture information of the user's head at different time points, and determines the first motion trajectory of the simulation component in the preset time period according to the posture information of the reference point at different time points.

[0075] S36, after determining the first motion trajectory according to step S32 or S35, the controller 122 sends a request to the camera module 124 of the delay detection device 120 to obtain a display image of the display device during the period when the simulated component moves according to the first motion trajectory.

[0076] S37 , after receiving the request for acquiring the display image, the camera module 124 sends the display image of the display device during the period when the simulated component moves along the first motion trajectory to the controller 122 .

[0077] S38, after receiving the above display image, the controller 122 obtains a second action sequence according to the display image, and determines a second motion trajectory of a preset time period according to the second action sequence.

[0078] S39, after determining the second motion trajectory, the controller 122 determines a delay of the display device in a preset time period based on the first motion trajectory and the second motion trajectory.

[0079] In other embodiments, the delay detection device 120 may have a collection module (not shown). The collection module may be used to obtain the first action sequence and send the first action sequence to the controller 122 of the delay detection device 120. The collection module may also be used to obtain the position information of multiple points to be measured at different time points during the user's action corresponding to the first action sequence, and send the position information to the controller 122 of the delay detection device 120.

[0080] Fig.12 FIG. 1 is a schematic diagram of the interaction between a display device, a delay detection device and a server 130 according to an exemplary embodiment of the present disclosure. Fig.12 To describe the specific testing process of the delay of the display device.

[0081] S41 , the display device 110 sends a first action sequence to the server 130 .

[0082] S42, after receiving the first action sequence, the server 130 determines whether there is a target action sequence corresponding to the first action sequence in the database. In response to the database having a target action sequence corresponding to the first action sequence, a target motion trajectory corresponding to the target action sequence is determined as the first motion trajectory of the simulation component.

[0083] S43, in response to the database not having a target action sequence corresponding to the first action sequence, the server 130 sends a request to the display device 110 to obtain the posture information of the user's head at different time points during the user's action corresponding to the first action sequence.

[0084] S44, after receiving the request for obtaining the posture information, the display device 110 sends the posture information of the user's head at different time points during the user's action corresponding to the first action sequence to the server 130.

[0085] S45, after receiving the above-mentioned acquired posture information, the server 130 determines the posture information of the reference point at different time points according to the posture information of the user's head at different time points, and determines the first motion trajectory of the simulation component in the preset time period according to the posture information of the reference point at different time points.

[0086] S46, after determining the first motion trajectory according to step S42 or S45, the server 130 sends a request to the controller 122 of the delay detection device 120 to obtain a display image of the display device during the period when the simulated component moves according to the first motion trajectory.

[0087] S47 , after receiving the request for acquiring the display image, the controller 122 sends the display image of the display device during the period when the simulated component moves along the first motion trajectory to the server 130 .

[0088] S48, after receiving the above display image, the server 130 obtains a second action sequence according to the display image, and determines a second motion trajectory of a preset time period according to the second action sequence.

[0089] S49, after determining the second motion trajectory, the server 130 determines a delay of the display device in a preset time period based on the first motion trajectory and the second motion trajectory.

[0090] In other embodiments, the delay detection device 120 may have a collection module (not shown). The collection module may be used to obtain the first action sequence and send the first action sequence to the server 130. The collection module may also be used to obtain the position information of the multiple points to be measured at different time points during the user's action corresponding to the first action sequence, and send the position information to the server 130.

[0091] An exemplary embodiment of the present disclosure also provides a delay detection device, which may include a simulation component, a drive motor, a camera module and a controller. The simulation component can be used to simulate the user's actions. The simulation component can be, for example, a simulated head model. The drive motor can be used to drive the simulation component to move, for example, drive the simulation component to move according to a first motion trajectory, wherein the first motion trajectory can be determined based on a first motion sequence, and the first motion sequence can include first time information and first position information. The camera module can be used to capture images, for example, to capture the display image of the display device to be tested during the period when the simulation component moves according to the first motion trajectory. The controller can be, for example, connected to the drive motor and the camera module by communication. Among them, the controller can be configured to: determine a second motion trajectory of a preset time period according to the displayed image, and determine the delay of the display device to be tested in the preset time period based on the first motion trajectory and the second motion trajectory.

[0092] In other examples, the second motion trajectory may be determined based on a second motion sequence acquired from the displayed image, and the second motion sequence may include second time information and second posture information.

[0093] In other examples, the controller may acquire a first motion trajectory of a preset time period according to the first action sequence, and control the drive motor to drive the simulation component to move according to the first motion trajectory.

[0094] It should be noted that the controller of the delay detection device can execute the above-mentioned delay test method 200.

[0095] An exemplary embodiment of the present disclosure also provides an electronic device, which includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned delay testing method 200.

[0096] The exemplary embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the delay test method 200 described above is implemented.

[0097] Fig.13 Schematically shows a block diagram of an electronic device 500 suitable for implementing an exemplary embodiment of the present disclosure. Fig.13As shown, the electronic device 500 includes a processor 501, which can perform various appropriate steps and processes according to computer program instructions stored in a read-only memory (ROM) 502 or computer program instructions loaded from a memory 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0098] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506; an output unit 507; a memory 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0099] The processor 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 501 may execute the various methods and processes described above, such as executing the above-mentioned delay test method 200. For example, in some embodiments, the above-mentioned delay test method 200 may be implemented as a computer software program, which is stored in a machine-readable medium, such as a memory 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the processor 501, the delay test method 200 described above may be executed. Alternatively, in other embodiments, the processor 501 may be configured to execute the above-mentioned delay test method 200 in any other appropriate manner (e.g., by means of firmware).

[0100] Various aspects of the present disclosure are described herein with reference to flowcharts and / or step diagrams of methods, devices (systems), and computer program products according to exemplary embodiments of the present disclosure. It should be understood that each step of the flowchart and / or step diagram and the combination of the steps in the flowchart and / or step diagram can be implemented by computer-readable program instructions.

[0101] These computer-readable program instructions can be provided to a processor, a general-purpose computer, a special-purpose computer, or a processing unit of other programmable data processing devices, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / steps specified in one or more steps in the flowchart and / or step diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / steps specified in one or more steps in the flowchart and / or step diagram.

[0102] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / steps specified in one or more steps in the flowchart and / or step diagram.

[0103] The flow chart and step diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each step in the flow chart or step diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the steps can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each step in the step diagram and / or flow chart, and the combination of the steps in the step diagram and / or flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] The above description is only an exemplary embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of protection involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the technical concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A delay test method, characterized in that: include: In response to receiving a first action sequence, obtaining a first motion trajectory of the simulation component in a preset time period according to the first action sequence, wherein the first action sequence includes first time information and first posture information; Acquire a second action sequence according to a display image of a display device during the period when the simulation component moves according to the first motion trajectory, and determine a second motion trajectory of the preset time period according to the second action sequence, wherein the second action sequence includes second time information and second posture information; and The delay of the display device in the preset time period is determined based on the first motion trajectory and the second motion trajectory.

2. The method according to claim 1, wherein: Acquiring a first motion trajectory of a simulation component in a preset time period according to the first action sequence includes: In response to the database having a target action sequence corresponding to the first action sequence, a target motion trajectory corresponding to the target action sequence is determined as the first motion trajectory.

3. The method according to claim 1, wherein: Acquiring a first motion trajectory of a simulation component in a preset time period according to the first action sequence includes: In response to the database not having a target action sequence corresponding to the first action sequence, acquiring position information of a reference point at different time points during a user performing an action corresponding to the first action sequence; and The first motion trajectory is determined based on the position and posture information of the reference point at different time points.

4. The method according to claim 3, wherein: Acquiring position information of the reference point at different time points during the user performing an action corresponding to the first action sequence, including: Acquire position information of a plurality of points to be measured at different time points during the time when the user performs an action corresponding to the first action sequence; and The position and posture information of the reference point at different time points is determined according to the position and posture information of the plurality of points to be measured at different time points, wherein the reference point is the center point of the three-dimensional space formed by the plurality of points to be measured.

5. The method according to claim 3, wherein: Determining the first motion trajectory based on the position information of the reference point at different time points includes: Determine the pose information of the simulation component at different time points according to the pose information of the reference point at different time points; Determining motion parameters of the simulation component at different time points according to the posture information of the simulation component at different time points, wherein the motion parameters include one or more of the rotation angle, speed and acceleration of the simulation component; and A third motion trajectory is generated based on the posture information and motion parameters of the simulation component at different time points, and the third motion trajectory is determined as the first motion trajectory.

6. The method according to any one of claims 1 to 5, wherein: Acquiring a second action sequence according to a display image of a display device during the period when the simulation component moves along the first motion trajectory, comprising: Performing feature extraction on the displayed image to obtain feature points, and determining the two-dimensional coordinates of the feature points at different time points; Determine the three-dimensional coordinates of the feature points at different time points according to the identity identifiers of the feature points, and construct two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature points at different time points; and The second action sequence is determined based on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature points at different time points.

7. The method according to claim 6, wherein: Determining the second action sequence based on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature points at different time points includes: Performing pose calculation on the two-dimensional coordinate-three-dimensional coordinate matching point pairs of the feature points at different time points to obtain the rotation matrix of the display image at different time points relative to the world coordinate system; and The second action sequence is determined based on the rotation matrix at different time points.

8. The method according to claim 6, wherein: Before extracting features from the displayed image to obtain feature points, the method further includes: Gaussian blur processing is performed on the displayed image.

9. The method according to any one of claims 1 to 5, wherein: Determining the delay of the display device in the preset time period based on the first motion trajectory and the second motion trajectory includes: performing linear fitting and / or quadratic curve fitting on the first motion trajectory and the second motion trajectory; and The delay of the display device in the preset time period is determined according to the fitted first motion trajectory and the second motion trajectory.

10. The method according to claim 9, wherein: Performing linear fitting and / or quadratic curve fitting on the first motion trajectory and the second motion trajectory includes: identifying linear intervals and nonlinear intervals of the first motion trajectory and the second motion trajectory; and A linear fitting is performed on the linear interval, and a quadratic curve fitting is performed on the nonlinear interval.

11. The method according to claim 9, wherein: The first motion trajectory and the second motion trajectory are time-position curves, and determining the delay of the display device in the preset time period according to the fitted first motion trajectory and the second motion trajectory includes: Obtaining a delay curve of the display device according to a time difference between the second motion trajectory and the first motion trajectory fitted under the same posture; and The delay of the display device in the preset time period is determined based on the delay curve of the display device.

12. The method according to claim 11, wherein: The time-posture curve is a time-posture angle curve, and the delay curve of the display device is obtained according to the time difference between the second motion trajectory and the first motion trajectory after fitting under the same posture, including: The delay curve of the display device is obtained according to the time difference between the second motion trajectory and the first motion trajectory after fitting at the same attitude angle, wherein the attitude angle includes a roll angle, a yaw angle, and a pitch angle.

13. A time delay detection device, characterized in that: include: Simulation components, used to simulate user actions; A driving motor, configured to drive the simulation component to move along a first motion trajectory, wherein the first motion trajectory is determined based on a first action sequence including first time information and first posture information; a camera module, used for capturing a display image of the display device under test when the simulation component moves along the first motion trajectory; and A controller is connected to the driving motor and the camera module for communication. Wherein, the controller is configured to: determine a second motion trajectory of a preset time period according to the display image, and determine the delay of the display device to be tested in the preset time period based on the first motion trajectory and the second motion trajectory, wherein the second motion trajectory is determined based on a second action sequence obtained from the display image, and the second action sequence includes second time information and second posture information.

14. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the delay testing method according to any one of claims 1 to 12.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the delay testing method according to any one of claims 1 to 12 is implemented.

Images