Virtual-real superposition accuracy test method and system and storage medium

Abstract

The application discloses a kind of virtual-real superposition precision test method, system and storage medium, it is related to wearable technology field.The method comprises the following steps: controlling the head-mounted display device to be measured to be in different test poses in turn;Get the first image pair obtained by the image of the display picture and the calibration plate of the double-camera acquisition head-mounted display device to be measured in test pose, according to the first space position of the real mark in the virtual-real pair in the first image in the camera coordinate system and the reference virtual-real relative pose corresponding to the virtual-real pair, determine the reference space position of the virtual mark in the virtual-real pair in the camera coordinate system, according to the second space position and reference space position of the virtual mark in the virtual-real pair in the camera coordinate system, determine the virtual-real superposition precision test result corresponding to the camera of the head-mounted display device to be measured in test pose;According to the virtual-real superposition precision test result of the double-camera of the head-mounted display device to be measured in different test poses, determine the virtual-real superposition precision test result of the head-mounted display device to be measured.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of wearable technology, and more particularly, to a virtual-real superposition accuracy testing method, a virtual-real superposition accuracy testing system, and a computer readable storage medium. BACKGROUND

[0002] When rendering virtual content, a head-mounted display device (AR / VR / MR) first reconstructs its pose in the real space through a camera-IMU, and then renders virtual content based on the pose. If the alignment error between the real coordinate system and the virtual coordinate system is too large, the virtual object will drift or be misplaced, so it is necessary to test the virtual-real superposition accuracy to ensure that the rendering position strictly corresponds to the real reference.

[0003] Currently, existing virtual-real superposition accuracy testing methods mainly rely on the following two methods:

[0004] (1) Subsystem unit testing: Parameters are calibrated and errors are measured for independent components such as optical display modules, IMUs, and SLAM algorithms. Since the errors of each subsystem are coupled and superimposed after the integration of the whole machine, unit testing cannot reflect the virtual-real superposition accuracy error in the real use environment.

[0005] (2) Artificial subjective evaluation: The tester wears the device and observes the superposition accuracy of the virtual marker and the real marker by eye to score. This method is not only low in efficiency and poor in repeatability, but also cannot provide quantitative data, making it difficult to locate specific defect modules.

[0006] Therefore, the above two virtual-real superposition accuracy testing methods have obvious deficiencies in testing efficiency, quantitative accuracy, and systematic defect positioning. SUMMARY

[0007] An object of embodiments of the present application is to provide a new technical solution for virtual-real superposition accuracy testing to solve the problems of low testing efficiency and poor testing accuracy in the prior art when testing the virtual-real superposition accuracy of a head-mounted display device.

[0008] According to a first aspect of the present application, a virtual-real superposition accuracy testing method is provided, comprising:

[0009] controlling a to-be-tested head-mounted display device to be in different test poses in turn;

[0010] For any test pose, obtaining a first image pair of images of a display screen and a calibration board of the to-be-tested head-mounted display device in the test pose captured by a dual-camera, and determining a virtual-real superposition accuracy test result of the to-be-tested head-mounted display device in the test pose according to the first image pair; wherein the relative pose of the dual-camera and the to-be-tested head-mounted display device is unchanged;

[0011] determine the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the different test poses;

[0012] The first image in the first image pair includes at least one virtual-to-real pair, and the virtual-to-real pair includes one real marker and one virtual marker. The determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose according to the first image pair includes:

[0013] For the first image corresponding to any camera in the first image pair, determining a reference spatial position of the virtual marker in the virtual-to-real pair in the camera coordinate system according to a first spatial position of the real marker in the virtual-to-real pair in the camera coordinate system and a reference virtual-to-real relative pose corresponding to the virtual-to-real pair, and determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera according to a second spatial position of the virtual marker in the virtual-to-real pair in the camera coordinate system and the reference spatial position.

[0014] The determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the dual cameras.

[0015] Optionally, the determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera according to the second spatial position of the virtual marker in the virtual-to-real pair in the camera coordinate system and the reference spatial position includes:

[0016] In a case where a distance between the second spatial position and the reference spatial position is less than a distance threshold, determining that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera is qualified.

[0017] In a case where the distance between the second spatial position and the reference spatial position is greater than or equal to the distance threshold, determining that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera is unqualified.

[0018] And / or, the determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the dual cameras includes:

[0019] determining that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose is qualified.

[0020] determining that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose is unqualified.

[0021] Optionally, before the to-be-tested head-mounted display device is controlled to be in different test poses in sequence, the method further comprises:

[0022] obtaining a second image obtained by the camera capturing an image of the to-be-tested head-mounted display device displaying a picture in a preset all-view pose and the calibration board; wherein the second image comprises the virtual-to-real pair.

[0023] determining a real space position of the real marker in the camera coordinate system according to a pixel coordinate of the real marker in the virtual-to-real pair in the second image;

[0024] determining a virtual space position of the virtual marker in the camera coordinate system according to a pixel coordinate of the virtual marker in the virtual-to-real pair in the second image;

[0025] determining the reference virtual-to-real relative pose according to the real space position and the virtual space position.

[0026] Optionally, before the to-be-tested head-mounted display device is controlled to be in different test poses in sequence, the method further comprises:

[0027] controlling the to-be-tested head-mounted display device to move along a preset trajectory, so as to initialize a pose detection system of the to-be-tested head-mounted display device.

[0028] Optionally, the obtaining of the first image pair obtained by the dual cameras capturing an image of the to-be-tested head-mounted display device displaying a picture in the test pose and the calibration board comprises:

[0029] obtaining a first sub-image pair obtained by the dual cameras capturing an image of the calibration board in the test pose;

[0030] obtaining a second sub-image pair obtained by the dual cameras capturing an image of the to-be-tested head-mounted display device displaying a picture in the test pose;

[0031] determining the first image pair according to the first sub-image pair and the second sub-image pair.

[0032] Optionally, after determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device, the method further comprises:

[0033] In a case where the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device is unqualified, performing virtual-to-real overlay accuracy adjustment processing on the to-be-tested head-mounted display device.

[0034] According to a second aspect of the present application, a virtual-to-real overlay accuracy test system is provided, comprising a mechanical arm, a dual camera, a clamp, a calibration board, a memory and a processor, the end of the mechanical arm is provided with the clamp, the clamp is used to fix a to-be-tested head-mounted display device and the dual camera, the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to perform control on the mechanical arm to drive the to-be-tested head-mounted display device and the dual camera to be in different test poses in turn, the dual camera is used to acquire a first image pair of a displayed picture of the to-be-tested head-mounted display device and an image of the calibration board under any test pose, and the processor is used to call the computer instructions from the memory to perform the method according to the first aspect.

[0035] Optionally, the mechanical arm drives the to-be-tested head-mounted display device and the dual camera to move around the center of the calibration board by a preset angle in parallel to the plane of the calibration board, so that the to-be-tested head-mounted display device and the dual camera are in different test poses in turn; wherein the preset angle is a ratio between 360° and the total number of the test poses.

[0036] Optionally, the calibration board is provided with a white circular mark as a real mark at each corner, and the area of the calibration board except the real mark is black.

[0037] Optionally, the clamp is used to control at least one of the to-be-tested head-mounted display device and the dual camera to move, so as to adjust the relative position between the to-be-tested head-mounted display device and the dual camera.

[0038] According to a third aspect of the present application, a computer readable storage medium is provided, which is optionally stored with a computer program, the computer program realizes the method according to the first aspect when executed by a processor.

[0039] An advantage of the present application is that the virtual-real overlay precision test in the test pose is automatically performed based on the first image pair obtained by the dual cameras collecting the image of the display picture of the head-mounted display device to be tested in the test pose and the image of the calibration board. Since the first image pair is obtained according to the image of the display picture of the head-mounted display device to be tested in the test pose and the image of the calibration board in the test pose, it reflects the actual virtual-real overlay in the test pose, so the test based on the first image pair can avoid the defect that the unit test cannot reflect the virtual-real overlay error in the real use environment, improve the accuracy of the virtual-real overlay precision test, and also avoid the problems of low test efficiency and low test accuracy caused by manual subjective evaluation, improve the accuracy and test efficiency of the virtual-real overlay precision test. In addition, based on the virtual-real overlay precision test results in different test poses, the virtual-real overlay precision of the head-mounted display device to be tested is evaluated, which can further improve the accuracy and test efficiency of the test. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and, together with the description, serve to explain the principles of the application.

[0041] Figure 1 is a block diagram of a hardware configuration of a virtual-real overlay precision test system according to an embodiment of the present application;

[0042] Figure 2 is a flowchart of a virtual-real overlay precision test method according to an embodiment of the present application;

[0043] Figure 3 is the relative position of the virtual marker mapping point and the real marker of the head-mounted display device to be tested in a preset all-view pose according to an example of the present application;

[0044] Figure 4 is the relative position of the virtual marker mapping point and the real marker of the head-mounted display device to be tested in a test pose according to an example of the present application. DETAILED DESCRIPTION

[0045] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of the components and steps set forth in these embodiments are not limiting to the scope of the present application unless otherwise specifically stated.

[0046] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way limiting to the scope of the application or its applications or uses.

[0047] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art(s) can not be discussed in detail herein. However, where appropriate, the techniques, methods, and apparatus should be considered as being part of the specification.

[0048] In all of the examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Thus, other examples of the exemplary embodiments can have different values.

[0049] It should be noted that like reference numerals and letters refer to like items throughout the drawings, and once an item is defined in one drawing, it is not necessary to discuss it further in subsequent drawings.

[0050] Figure 1 is a block diagram of a hardware configuration of a virtual-real overlay precision test system 1000 provided according to an embodiment of the present application.

[0051] The virtual-real overlay precision test system 100 includes a mechanical arm 11, a dual camera 12, a clamp 13, a calibration board 14, a memory 15, and a processor 16.

[0052] The end of the mechanical arm 11 is provided with the clamp 13, which is used to fix the to-be-tested head-mounted display device and the dual camera 12. The memory 15 is used to store computer instructions, and the processor 16 is used to call the computer instructions from the memory 15 to execute control of the mechanical arm 11 to drive the to-be-tested head-mounted display device and the dual camera 12 to be in different test poses in sequence. The dual camera 12 is used to acquire a first image pair of an image of the calibration board 14 and a displayed picture of the to-be-tested head-mounted display device in any test pose. The processor 16 is used to call the computer instructions from the memory 15 to execute the virtual-real overlay precision test method of the present application.

[0053] In the present embodiment, the mechanical arm 11 adopts a multi-joint series design, and the mechanical arm 11 movement covers six degrees of freedom pose changes to simulate the wearer's head multi-directional dynamic observation.

[0054] The end of the mechanical arm 11 is integrated with the high-precision clamp 13, which is used to fix the to-be-tested head-mounted display device and the dual camera 12.

[0055] The to-be-tested head-mounted display device can be, for example, an AR device, an MR device, a VR device, etc., which is not limited herein.

[0056] In an embodiment of the present application, the to-be-tested head-mounted display device is an MR device. Since the field of view angle of the MR device is larger and the display precision is higher, the dual camera can be replaced by two high-precision, large-view-angle RGB cameras.

[0057] The two cameras 12 are symmetrically arranged behind the display screen of the head-mounted display device to be tested to capture the image displayed by the head-mounted display device to be tested.

[0058] In an embodiment of the present application, to avoid the test error caused by the misalignment of the optical axis of the camera and the optical center of the head-mounted display device to be tested when the two cameras capture the first image pair, the optical axis of the two cameras 12 can be aligned with the optical center of the head-mounted display device to be tested.

[0059] During the whole test process, the relative pose of the two cameras 12 and the head-mounted display device to be tested is unchanged.

[0060] In an embodiment of the present application, the clamp 13 is used to control the movement of at least one of the head-mounted display device to be tested and the two cameras 12 to adjust the relative position between the head-mounted display device to be tested and the two cameras 12.

[0061] In the embodiment, to facilitate the alignment of the optical axis of the two cameras 12 and the optical center of the head-mounted display device to be tested, at least one of the two clamps used to fix the head-mounted display device to be tested and the two cameras 12 can be a movable clamp. Then the relative position between the head-mounted display device to be tested and the two cameras 12 is adjusted by controlling the movement of the movable clamp.

[0062] The calibration plate 14 is arranged on one side of the mechanical arm 11 to facilitate the two cameras 12 fixed on the mechanical arm 11 to capture the image of the calibration plate.

[0063] The mechanical arm 11 drives the head-mounted display device to be tested and the two cameras 12 to be in different test poses in turn. Since the two cameras 12 not only capture the image displayed by the head-mounted display device to be tested in the test pose, but also capture the image of the calibration plate 14 in the test pose, the first image pair is obtained. Therefore, different test poses need to be determined based on the position of the calibration plate 14.

[0064] In an embodiment of the present application, the mechanical arm 11 drives the head-mounted display device to be tested and the two cameras 12 to move around the center of the calibration plate 14 by a preset angle in a plane parallel to the calibration plate 14, so that the head-mounted display device to be tested and the two cameras 12 are in different test poses in turn.

[0065] In the embodiment, different test poses combined together need to meet the requirement of one round around the calibration plate 14. The center of the calibration plate 14 refers to the geometric center of the calibration plate 14. The preset angle is the ratio between 360° and the total number of test poses.

[0066] In an example, the total number of test poses is 10, and the preset angle is 36°.

[0067] In an embodiment of the present application, the four corners of the calibration board 14 are respectively provided with a white circular mark as a real mark, and the rest of the calibration board 14 is black.

[0068] In the embodiment, as shown in FIG. 2, the four corners of the calibration board 14 are respectively provided with a large circle (i.e., four real marks), and the rest of the calibration board 14 is black. Figure 1 If the virtual marks generated by the head-mounted display device to be tested are projected into the three-dimensional space, the mapping points of the virtual marks in the three-dimensional space are distributed in the central black background area of the calibration board 14. In other words, the central black background area of the calibration board 14 is used to project the mapping points of the virtual marks generated by the head-mounted display device to be tested in the three-dimensional space (i.e., the small circles in the 4x5 array in the middle of FIG. 2 are the mapping points of the virtual marks in the three-dimensional space). Figure 1

[0069] By using the anti-scattering black-and-white anti-color design calibration board, the scattering interference of the optical waveguide can be effectively suppressed.

[0070] The following takes the head-mounted display device to be tested as an AR device as an example to describe the process of collecting a first image pair by the dual cameras in a test pose:

[0071] In the case that the mechanical arm 11 fixes the AR device and the dual cameras 12, and the optical axis of the dual cameras 12 is aligned with the optical center of the AR device, the mechanical arm 11 drives the AR device and the dual cameras 12 to move 36° around the center of the calibration board 14 in a plane parallel to the calibration board 14, so that the AR device and the dual cameras 12 are in a test pose. In the test pose, the AR device will generate and display virtual marks based on the detected pose, and will also display real marks based on the real marks on the calibration board collected by itself, that is, the screen displayed by the AR device at this time includes real marks and virtual marks. At this time, the dual cameras collect the screen displayed by the AR device, and a first image pair is obtained. In another case, if the AR device only displays virtual marks and does not display real marks, the dual cameras can collect real marks on the calibration board through the AR device while collecting the screen displayed by the AR device, so as to obtain a first image pair.

[0072] The following takes the head-mounted display device to be tested as a VR device as an example to describe the process of collecting a first image pair by the dual cameras in a test pose:

[0073] ​In the case that the mechanical arm 11 fixes the VR device and the dual camera 12, and the optical axis of the dual camera 12 is aligned with the optical center of the AR device, the mechanical arm 11 drives the VR device and the dual camera 12 to move 36° around the center of the calibration plate 14 in a plane parallel to the calibration plate 14, so that the VR device and the dual camera 12 are in a test pose. In the test pose, the VR device generates and displays a virtual marker based on the pose detected by itself, at this time, the dual camera captures the display screen of the VR device to obtain a first sub-image pair. Then, the VR device is removed, and the mechanical arm 11 is controlled to make the dual camera 12 in the test pose to obtain a second sub-image pair. Finally, the first sub-image pair and the second sub-image pair are spliced according to the corresponding cameras to obtain a first image of the corresponding camera, and then a first image pair is obtained.

[0074] The processor is configured to invoke the computer instructions from the memory to execute a virtual-real superimposition accuracy test method provided by the embodiments of the present application to evaluate the virtual-real superimposition accuracy of the head-mounted display device to be tested.

[0075] When rendering virtual content, the head-mounted display device (AR / VR / MR) first reconstructs its pose in the real space through the camera-IMU, and then renders virtual content based on the pose.

[0076] Ideally, after the head-mounted display device displays the generated virtual marker on the display screen, no matter how the pose of the head-mounted display device changes, the mapping point of the virtual marker in the three-dimensional space is fixed. For ease of description, the mapping point of the virtual marker in the three-dimensional space is referred to as a virtual marker mapping point. However, in the actual environment, the optical-IMU fusion positioning of the head-mounted display device has drift and noise, detection delay, lens distortion, calibration error, mechanical assembly tolerance, and the like, which will cause the projection matrix of the virtual marker to deviate from the ideal value when generating virtual content, resulting in a shift in the position of the virtual marker mapping point. Therefore, the virtual-real superimposition accuracy of the head-mounted display device can be evaluated based on the degree of position shift of the virtual marker mapping point in multiple test poses.

[0077] The present application provides a virtual-real superimposition accuracy test method, which is applied to a virtual-real superimposition accuracy test system 100 as shown in Figure 1 The memory and the processor in the virtual-real superimposition accuracy test system 100 as shown in Figure 2 The method comprises the following steps S2100 to S2300.

[0078] In step S2100, the head-mounted display device to be tested is controlled to be in different test poses in sequence.

[0079] In the present embodiment, the head-mounted display device to be tested can be an AR device, an MR device, a VR device, and the like, which is not limited herein.

[0080] The test pose can be a pose set by a tester according to test requirements, and the specific position and direction of the test pose are not limited herein.

[0081] In the virtual-real superimposition accuracy test system shown in FIG. 1, the step S2100 of controlling the to-be-tested head-mounted display device to be in different test poses in sequence can include: controlling the mechanical arm to drive the to-be-tested head-mounted display device and the dual cameras to be in different test poses in sequence. Figure 1 The to-be-tested head-mounted display device and the dual cameras are controlled to be in different test poses in sequence.

[0082] Since the dual cameras not only need to collect the display screen of the to-be-tested head-mounted display device in the test pose, but also need to collect the image of the calibration board in the test pose, a first image pair is obtained. Therefore, the different test poses need to be determined based on the position of the calibration board.

[0083] In an embodiment of the present application, the step S2100 of controlling the to-be-tested head-mounted display device to be in different test poses in sequence includes: controlling the mechanical arm to drive the to-be-tested head-mounted display device and the dual cameras to move around the center of the calibration board by a preset angle in a plane parallel to the calibration board, so that the to-be-tested head-mounted display device and the dual cameras are in different test poses in sequence.

[0084] In the embodiment, the different test poses in combination need to meet the requirement of one round around the calibration board. The center of the calibration board refers to the geometric center of the calibration board. The preset angle is a ratio between 360° and the total number of test poses.

[0085] In an example, the total number of test poses is 10, and the preset angle is 36°

[0086] In an embodiment of the present application, before the step S2100 of controlling the to-be-tested head-mounted display device to be in different test poses in sequence, the method further includes a step S2000.

[0087] The step S2000 controls the to-be-tested head-mounted display device to move along a preset trajectory, so that a pose detection system of the to-be-tested head-mounted display device is initialized.

[0088] In the embodiment, the preset trajectory can be a trajectory designed for the initialization of the pose detection system of the to-be-tested head-mounted display device, and the specific shape of the preset trajectory is not limited herein.

[0089] The pose detection system of the to-be-tested head-mounted display device includes at least one of an IMU (Inertial Measurement Unit) and a SLAM (Simultaneous Localization and Mapping), and the initialization of the pose detection system means that the pose detection system converges, thereby improving the pose control accuracy of the to-be-tested head-mounted display device.

[0090] In step S2200, for any test pose, a first image pair captured by the dual cameras from the image of the display screen of the to-be-tested head-mounted display device and the calibration board in the test pose is obtained, and a virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose is determined according to the first image pair.

[0091] The relative pose between the dual cameras and the to-be-tested head-mounted display device is unchanged.

[0092] Taking the to-be-tested head-mounted display device as an AR device as an example, in any test pose, the AR device generates and displays a virtual marker based on the detected pose, and displays a real marker based on the captured real marker on the calibration board, that is, the display screen of the AR device at this time includes the real marker and the virtual marker. At this time, the dual cameras capture the display screen of the AR device to obtain the first image pair. In another case, if the AR device only displays the virtual marker and does not display the real marker, the dual cameras can capture the real marker on the calibration board through the AR device while capturing the display screen of the AR device, thereby obtaining the first image pair.

[0093] In an embodiment of the present application, the to-be-tested head-mounted display device is a VR / MR device.

[0094] Since the user cannot see the external real world when wearing the VR / MR device, that is, the dual cameras cannot capture the real marker on the calibration board and the display screen of the VR / MR device in a test pose at the same time.

[0095] Based on this, in an embodiment of the present application, the step S2200 of obtaining the first image pair captured by the dual cameras from the image of the display screen of the to-be-tested head-mounted display device and the calibration board in the test pose includes steps S2200.1-S2200.3.

[0096] In step S2200.1, a first sub-image pair captured by the dual cameras from the image of the calibration board in the test pose is obtained.

[0097] In this embodiment, in the case where the mechanical arm does not carry the VR / MR device, the mechanical arm is controlled to reach a test pose, and the dual cameras are used to capture the calibration board to obtain the first sub-image pair. The first sub-image pair is an image of the calibration board in the view angle corresponding to the test pose.

[0098] In step S2200.2, a second sub-image pair captured by the dual cameras from the image of the display screen of the to-be-tested head-mounted display device in the test pose is obtained.

[0099] In this embodiment, the mechanical arm carries the VR / MR device to the test pose, and the dual cameras capture the image of the VR / MR device displayed at this time to obtain a second image pair. The second image pair is the image of the virtual marker displayed by the VR / MR device at the test pose.

[0100] Step S2200.3, determining the first image pair according to the first image pair and the second image pair.

[0101] In this embodiment, the first image and the second image corresponding to each camera are combined to obtain the first image corresponding to the camera. Then, the first image pair corresponding to the dual cameras can be obtained.

[0102] In the example in which the calibration board includes 4 real markers and a 4x5 array of virtual marker mapping points, the first image obtained by any camera of the dual cameras capturing the image of the test pose of the head-mounted display device displayed at the test pose and the calibration board may include only one real marker and one virtual marker, may include 4 real markers and a 4x5 array of virtual markers (i.e., 20 real-virtual pairs), or may include 1 real marker and 2 virtual markers (i.e., two real-virtual pairs), which is not limited here.

[0103] That is, the first image includes at least one real-virtual pair. The real-virtual pair includes one real marker and one virtual marker.

[0104] In this embodiment, the determination of the virtual-real superimposition accuracy test result of the head-mounted display device at the test pose according to the first image pair in step S2200 includes steps S3100-S3200.

[0105] Step S3100, for the first image corresponding to any camera pair in the first image pair, determining the reference spatial position of the virtual marker in the real-virtual pair in the camera coordinate system according to the first spatial position of the real marker in the real-virtual pair in the camera coordinate system and the reference real-virtual relative pose corresponding to the real-virtual pair, and determining the virtual-real superimposition accuracy test result of the head-mounted display device at the test pose corresponding to the camera according to the second spatial position of the virtual marker in the real-virtual pair in the camera coordinate system and the reference spatial position.

[0106] In the embodiment, for each camera in the dual cameras, the camera coordinate system thereof is a coordinate system established with the optical axis of the camera as the origin. Since the dual cameras are carried on the robot arm together with the head-mounted display device to be tested, during the movement of the robot arm with the head-mounted display device to be tested, the positions of the dual cameras in the three-dimensional space also change accordingly, and the camera coordinate systems corresponding to the two cameras also change, so that the three-dimensional space positions of the real marker and the virtual marker in the camera coordinate systems also change. Therefore, each time the head-mounted display device to be tested is moved, the first space position of the real marker in the camera coordinate system is calculated based on the pixel coordinates of the real marker in the first image captured by any camera in the current test pose and the back projection parameters of the camera, and the second space position of the virtual marker in the camera coordinate system is calculated based on the pixel coordinates of the virtual marker in the first image and the back projection parameters of the camera.

[0107] Based on this, in one embodiment of the present application, before step S3100 is performed, the method further comprises step SA.

[0108] In step SA, for the first image corresponding to any camera in the first image pair, the first space position of the real marker in the camera coordinate system is determined according to the pixel coordinates of the real marker in the real-virtual pair of the first image and the back projection parameters of the camera, and the second space position of the virtual marker in the camera coordinate system is determined according to the pixel coordinates of the virtual marker in the real-virtual pair of the first image and the back projection parameters of the camera.

[0109] In the embodiment, the back projection parameters can include the extrinsic parameters and the intrinsic parameters of the camera.

[0110] The back projection parameters can be determined according to the attributes of the camera, or can be determined after the calibration of the back projection parameters of the camera before the current real-virtual superposition accuracy test, which is not limited here.

[0111] In one embodiment of the present application, before step S2100 is performed to control the head-mounted display device to be tested to be in different test poses in turn, the method further comprises steps SA1 to SA2. That is, the back projection parameters of the camera in step SA are determined through steps SA1 to SA2.

[0112] In step SA1, a set of calibration images obtained by the camera in different calibration poses capturing the calibration board are acquired.

[0113] In this embodiment, without fixing the head-mounted display device under test to the robotic arm, the robotic arm is controlled to move the camera sequentially to each of the different calibration poses to acquire calibration images of the calibration board from the viewpoint corresponding to that calibration pose, thus obtaining a calibration image set. Different calibration poses can be the same as or different from different test poses; this is not limited here.

[0114] Step SA2: Determine the back projection parameters of the camera based on the calibration image set.

[0115] In this embodiment, corner detection is performed on each calibration image in the calibration image set. Based on the detected corners, the extrinsic and intrinsic parameters of the camera are determined as the back projection parameters of the camera.

[0116] The reference virtual-real relative pose reflects the ideal relative spatial position relationship of the virtual-real pair in the camera coordinate system.

[0117] For ease of explanation, Figure 1 The large circle in the upper left corner of the calibration plate is called the first true mark. Figure 1 The small circle closest to the top left corner (i.e., the small circle in the first row and first column of the 4x5 array) is called the first virtual marker, and these two form a virtual-real pair. Taking this virtual-real pair as an example, the virtual-real superposition accuracy testing method of this application is explained. That is, for this virtual-real pair, there exists a reference virtual-real relative pose corresponding to the left camera and a reference virtual-real relative pose corresponding to the right camera. The reference virtual-real relative pose reflects the ideal relative spatial position relationship of the virtual-real pair in the corresponding camera coordinate system.

[0118] The reference virtual-real relative pose can be obtained based on the relative spatial position relationship of the virtual-real pair in the corresponding camera coordinate system when the head-mounted display device is in an ideal state (i.e., ideal pose detection accuracy and no detection delay, etc.).

[0119] In one embodiment of this application, before step S2100 controls the head-mounted display device under test to be in different test poses in sequence, the method further includes steps S4100 to S4400.

[0120] Step S4100: Obtain a second image obtained by the camera capturing the image displayed by the head-mounted display device under the preset full-view pose and the image of the calibration plate.

[0121] The second image includes the virtual-real pair.

[0122] In this embodiment, the preset full-view pose can be a pose that allows the real mark on the calibration board to fall completely within the field of view of the camera.

[0123] The camera in the present step and the camera in step S3100 are corresponding, that is, if the camera in step S3100 is a left camera, the camera in the present step is a left camera. If the camera in step S3100 is a right camera, the camera in the present step is a right camera.

[0124] In one example, the second image includes a real-virtual pair composed of the first real marker 1 and the first virtual marker 2.

[0125] Step S4200, determining a real space position of the real marker in the camera coordinate system according to the pixel coordinates of the real marker in the real-virtual pair.

[0126] In the present embodiment, the real space position of the real marker in the camera coordinate system is determined based on the pixel coordinates of the real marker in the real-virtual pair and the back projection parameters of the camera.

[0127] Continuing the above example, in the example that the second image includes a real-virtual pair composed of the first real marker 1 and the first virtual marker 2, the real space position of the first real marker 1 in the camera coordinate system can be as shown in FIG. 4B. Figure 3

[0128] Step S4300, determining a virtual space position of the virtual marker in the camera coordinate system according to the pixel coordinates of the virtual marker in the real-virtual pair.

[0129] In the present embodiment, the virtual space position of the virtual marker in the camera coordinate system is determined based on the pixel coordinates of the virtual marker in the real-virtual pair and the back projection parameters of the camera. The virtual space position is the space position of the space mapping point of the virtual marker in the camera coordinate system.

[0130] Continuing the above example, in the example that the second image includes a real-virtual pair composed of the first real marker 1 and the first virtual marker 2, the virtual space position of the first virtual marker 2 in the camera coordinate system can be as shown in FIG. 4C. Figure 3

[0131] Step S4400, determining a reference real-virtual relative pose of the real-virtual pair according to the real space position and the virtual space position.

[0132] In the present embodiment, the reference real-virtual relative pose can be a reference rotation matrix and a reference translation matrix.

[0133] Continuing the above example, the reference real-virtual relative pose of the real-virtual pair is determined based on the relative position of the first real marker 1 and the first virtual marker 2 in Figure 3

[0134] ​​​The relative pose of the virtual-real pair corresponding to the reference virtual-real pair is fixed regardless of the movement of the robot arm. In other words, the relative position relationship between the real marker and the virtual marker mapping point on the calibration board in the virtual-real pair is fixed regardless of the movement of the robot arm, and therefore, based on the first spatial position of the real marker in the virtual-real pair in the camera coordinate system and the relative pose of the virtual-real pair corresponding to the reference virtual-real pair, the ideal spatial position of the virtual marker in the camera coordinate system, i.e., the reference spatial position, can be obtained.

[0135] Exemplarily, as shown in FIG. 1, in the test pose, the first spatial position of the first real marker 1 in the camera coordinate system is at the position shown in the figure, and the second spatial position of the first virtual marker 2 in the camera coordinate system is at the position shown in the figure (i.e., the actual spatial position of the first virtual marker). Based on the first spatial position of the first real marker 1 in the camera coordinate system and the relative pose of the virtual-real pair corresponding to the reference virtual-real pair, the reference spatial position of the first virtual marker in the camera coordinate system, i.e., the ideal spatial position of the first virtual marker, can be determined, as shown in the position corresponding to the marker 3 in FIG. 1. Figure 4 Figure 4

[0136] In an embodiment of the present application, the reference relative pose includes a reference rotation matrix and a reference translation matrix.

[0137] In this embodiment, the step S3100 of determining the reference spatial position of the virtual marker in the camera coordinate system in the virtual-real pair based on the first spatial position of the real marker in the virtual-real pair in the camera coordinate system and the relative pose of the virtual-real pair corresponding to the reference virtual-real pair includes:

[0138] determining the reference spatial position of the virtual marker in the camera coordinate system based on the first spatial position, the reference rotation matrix and the reference translation matrix.

[0139] In this embodiment, the reference spatial position of the virtual marker in the camera coordinate system can be obtained by rotating the first spatial position through the reference rotation matrix and translating it through the reference translation matrix.

[0140] Based on the second spatial position and the reference spatial position of the virtual marker in the camera coordinate system, a virtual-real superimposition accuracy test result of the head-mounted display device to be tested corresponding to the camera in the test pose is determined. The virtual-real superimposition accuracy test result is qualified or unqualified.

[0141] ​​In an embodiment of the present application, the step S3100 of determining the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device corresponding to the camera in the test pose according to the second spatial position of the virtual marker in the virtual-to-real pair in the camera coordinate system and the reference spatial position includes steps S3100.1-S3100.2.

[0142] The step S3100.1 includes: determining that the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device corresponding to the camera in the test pose is qualified when the distance between the second spatial position and the reference spatial position is less than the distance threshold.

[0143] In the embodiment, the distance threshold can be flexibly set according to the requirement of test accuracy, which is not limited herein.

[0144] The step S3100.2 includes: determining that the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device corresponding to the camera in the test pose is unqualified when the distance between the second spatial position and the reference spatial position is greater than or equal to the distance threshold.

[0145] The step S3200 includes: determining the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-to-real superimposition accuracy test results of the to-be-tested head-mounted display device corresponding to the dual cameras in the test pose.

[0146] In an embodiment of the present application, the step S3200 of determining the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-to-real superimposition accuracy test results of the to-be-tested head-mounted display device corresponding to the dual cameras in the test pose includes steps S3200.1 and S3200.2.

[0147] The step S3200.1 includes: determining that the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose is qualified when the virtual-to-real superimposition accuracy test results of the to-be-tested head-mounted display device corresponding to the dual cameras in the test pose are all qualified.

[0148] The step S3200.2 includes: determining that the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose is unqualified when the virtual-to-real superimposition accuracy test results of the to-be-tested head-mounted display device corresponding to the dual cameras in the test pose are unqualified.

[0149] Since the movement of the human head is complex and changeable, it is not limited to the virtual-real superposition accuracy test at one test pose, and therefore, in order to improve the accuracy of the virtual-real superposition accuracy test, the virtual-real superposition accuracy test can be performed at multiple test poses to accurately evaluate the pose detection accuracy of the to-be-tested head-mounted display device.

[0150] In step S2300, the virtual-real superposition accuracy test result of the to-be-tested head-mounted display device is determined according to the virtual-real superposition accuracy test results of the to-be-tested head-mounted display device at different test poses.

[0151] In the embodiment, when the virtual-real superposition accuracy test results of the to-be-tested head-mounted display device at different test poses are all qualified, it is determined that the virtual-real superposition accuracy test result of the to-be-tested head-mounted display device is qualified. When the virtual-real superposition accuracy test results of the to-be-tested head-mounted display device at different test poses are not qualified, it is determined that the virtual-real superposition accuracy test result of the to-be-tested head-mounted display device is not qualified.

[0152] For example, 10 test poses are set, and for each test pose, step S2200 is performed to obtain 10 virtual-real superposition accuracy test results corresponding to 10 test poses. When the 10 virtual-real superposition accuracy test results are all qualified, it is determined that the virtual-real superposition accuracy test result of the to-be-tested head-mounted display device is qualified. If there is an unqualified result in the 10 virtual-real superposition accuracy test results, it is determined that the virtual-real superposition accuracy test result of the to-be-tested head-mounted display device is not qualified.

[0153] In an embodiment of the present application, after the virtual-real superposition accuracy test result of the to-be-tested head-mounted display device is determined in step S2300, the method further comprises:

[0154] When the virtual-real superposition accuracy test result of the to-be-tested head-mounted display device is not qualified, the virtual-real superposition accuracy adjustment processing is performed on the to-be-tested head-mounted display device.

[0155] In the embodiment, the virtual-real superposition accuracy adjustment processing can be to detect the drift of the pose detection system of the to-be-tested head-mounted display device, to perform optical re-calibration on the to-be-tested head-mounted display device, to detect the perpendicularity error of the to-be-tested head-mounted display device, and to improve the virtual-real superposition accuracy of the to-be-tested head-mounted display device.

[0156] The first image pair obtained by collecting the image of the display picture of the to-be-tested head-mounted display device in the test pose and the image of the calibration board through the dual cameras is used to automatically perform the virtual-real overlay accuracy test in the test pose. Since the first image pair is obtained according to the image displayed by the to-be-tested head-mounted display device in the test pose and the image of the calibration board in the test pose, it reflects the actual virtual-real overlay in the test pose, and thus the test based on the first image pair can avoid the defect that the unit test cannot reflect the virtual-real overlay error in the real use environment, improve the accuracy of the virtual-real overlay accuracy test, and avoid the problems of low test efficiency and low test accuracy caused by manual subjective evaluation, thereby improving the accuracy and efficiency of the virtual-real overlay accuracy test. In addition, the virtual-real overlay accuracy of the to-be-tested head-mounted display device is evaluated based on the virtual-real overlay accuracy test results in different test poses, which can further improve the accuracy and efficiency of the test.

[0157] The virtual-real overlay accuracy test method of one embodiment of the present application will be described below with the to-be-tested head-mounted display device as an example of the to-be-tested AR device, which includes steps S1-S10.

[0158] In step S1, the dual cameras and the to-be-tested AR device are installed on the clamp of the mechanical arm, and the optical axis of the dual cameras and the optical center of the to-be-tested AR device are aligned.

[0159] In step S2, the to-be-tested AR device is removed from the clamp, and the mechanical arm is controlled to move to different calibration poses in sequence to obtain a set of calibration images collected by the camera in different calibration poses.

[0160] In this example, the calibration board includes four real markers.

[0161] In step S3, the back projection parameters of the camera are determined according to the set of calibration images.

[0162] In this example, the back projection parameters include camera extrinsic parameters and camera intrinsic parameters.

[0163] In step S4, the to-be-tested AR device is placed on the clamp, and the to-be-tested AR device is controlled to move along a preset trajectory to initialize the pose detection system of the to-be-tested AR device.

[0164] In step S5, the mechanical arm is controlled to move the to-be-tested AR device to a preset all-view pose, and a second image is collected by the camera.

[0165] The second image includes four real markers and a 4x5 array of virtual markers, i.e., there are 20 virtual-real pairs in the second image.

[0166] Step S6, for any one of the 20 virtual-real pairs, determine a reference virtual-real relative pose corresponding to the virtual-real pair, to obtain 20 reference virtual-real relative poses corresponding to the 20 virtual-real pairs.

[0167] Specifically, the reference virtual-real relative pose corresponding to a virtual-real pair is determined as follows: first, determine the real space position of the real marker in the camera coordinate system according to the pixel coordinates of the real marker in the virtual-real pair and the back projection parameters of the camera. Then, determine the virtual space position of the virtual marker in the camera coordinate system according to the pixel coordinates of the virtual marker in the virtual-real pair and the back projection parameters of the camera. According to the real space position and the virtual space position, the reference virtual-real relative pose of the virtual-real pair is determined.

[0168] Step S7, control the mechanical arm to drive the to-be-tested AR device and the dual camera to move 36° around the center of the calibration board in a plane parallel to the calibration board, so that the to-be-tested AR device and the dual camera are sequentially in 10 test poses.

[0169] Step S8, for any test pose, obtain the image pair of the first image pair of the image of the to-be-tested head-mounted display device displayed in the test pose and the calibration board captured by the dual camera, and determine the virtual-real superimposition accuracy test result of the to-be-tested AR device in the test pose according to the first image pair.

[0170] The first image in the first image pair includes 4 virtual-real pairs, and each virtual-real pair includes one real marker and one virtual marker. The following will illustrate how to determine the virtual-real superimposition accuracy test result of the to-be-tested AR device in a test pose according to the first image pair in the test pose through an example:

[0171] For any virtual-real pair in the first image corresponding to any camera in the first image pair, determine the reference space position of the virtual marker in the camera coordinate system in the virtual-real pair according to the first space position of the real marker in the camera coordinate system in the virtual-real pair and the reference virtual-real relative pose corresponding to the virtual-real pair. According to the second space position of the virtual marker in the camera coordinate system in the virtual-real pair and the reference space position, determine the virtual-real superimposition accuracy test result corresponding to the virtual-real pair. According to the virtual-real superimposition accuracy test results corresponding to the 4 virtual-real pairs in the first image, determine the virtual-real superimposition accuracy test result of the to-be-tested AR device corresponding to the camera in the test pose. According to the virtual-real superimposition accuracy test results of the to-be-tested AR device corresponding to the dual camera in the test pose, determine the virtual-real superimposition accuracy test result of the to-be-tested AR device in the test pose.

[0172] The virtual-real superimposition accuracy test result corresponding to the virtual-real pair is determined according to the second space position of the virtual marker in the camera coordinate system in the virtual-real pair and the reference space position, including:

[0173] In a case where the distance between the second spatial position and the reference spatial position is less than the distance threshold, it is determined that the virtual-real pair corresponds to a virtual-real superimposition accuracy test result that is qualified, and in a case where the distance between the second spatial position and the reference spatial position is greater than or equal to the distance threshold, it is determined that the virtual-real pair corresponds to a virtual-real superimposition accuracy test result that is unqualified.

[0174] The determining, according to the four virtual-real superimposition accuracy test results corresponding to the four virtual-real pairs in the first image, of a virtual-real superimposition accuracy test result of the AR device under test corresponding to the camera in the test pose includes:

[0175] In a case where the four virtual-real superimposition accuracy test results corresponding to the four virtual-real pairs are all qualified, it is determined that the virtual-real superimposition accuracy test result of the AR device under test corresponding to the camera in the test pose is qualified, and in a case where the four virtual-real superimposition accuracy test results corresponding to the four virtual-real pairs are unqualified, it is determined that the virtual-real superimposition accuracy test result of the AR device under test corresponding to the camera in the test pose is unqualified.

[0176] The determining, according to the virtual-real superimposition accuracy test result of the AR device under test corresponding to the dual cameras in the test pose, of the virtual-real superimposition accuracy test result of the AR device under test in the test pose includes:

[0177] In a case where the virtual-real superimposition accuracy test results of the AR device under test corresponding to the dual cameras in the test pose are all qualified, it is determined that the virtual-real superimposition accuracy test result of the AR device under test in the test pose is qualified, and in a case where the virtual-real superimposition accuracy test results of the AR device under test corresponding to the dual cameras in the test pose are unqualified, it is determined that the virtual-real superimposition accuracy test result of the AR device under test in the test pose is unqualified.

[0178] Step S9: In a case where the ten virtual-real superimposition accuracy test results corresponding to the ten test poses are all qualified, it is determined that the virtual-real superimposition accuracy test result of the AR device under test is qualified.

[0179] Step S10: In a case where the ten virtual-real superimposition accuracy test results corresponding to the ten test poses are unqualified, it is determined that the virtual-real superimposition accuracy test result of the AR device under test is unqualified.

[0180] The application also provides a computer readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the method of any of the above method embodiments.

[0181] The application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions embodied therewith to cause a processor to implement various aspects of the application.

[0182] Computer readable storage media can be tangible storage devices that can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0183] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0184] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0185] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0186] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0187] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0188] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0189] Embodiments of the application have been described above. The descriptions are intended to be illustrative, and not restrictive, of possible embodiments of the application. Many modifications and variations of the described embodiments are possible, given the benefit of the present disclosure, without departing from the scope and spirit of the described embodiments. The scope of the application is defined by the appended claims.

Claims

1. A method for testing the accuracy of virtual-real superposition, characterized in that, include: The head-mounted display device under test is controlled to be in different test positions sequentially; For any test pose, a first image pair is obtained by acquiring the image of the display screen and the calibration plate of the head-mounted display device under test acquired by the dual cameras in the test pose, and the virtual-real superposition accuracy test result of the head-mounted display device under test in the test pose is determined based on the first image pair; wherein, the relative pose of the dual cameras and the head-mounted display device under test remains unchanged. Based on the test results of the virtual-real superposition accuracy of the head-mounted display device under test in different test poses, the test results of the virtual-real superposition accuracy of the head-mounted display device under test are determined; Wherein, the first image in the first image pair includes at least one virtual-real pair, the virtual-real pair including a real marker and a virtual marker, and the step of determining the virtual-real superposition accuracy test result of the head-mounted display device under test in the test pose based on the first image pair includes: For any first image corresponding to any camera in the first image pair, the reference spatial position of the virtual marker in the virtual-real pair in the camera coordinate system is determined based on the first spatial position of the real marker in the virtual-real pair in the camera coordinate system and the reference virtual-real relative pose corresponding to the virtual-real pair. Furthermore, the virtual-real superposition accuracy test result of the head-mounted display device under test corresponding to the camera in the test pose is determined based on the second spatial position of the virtual marker in the virtual-real pair in the camera coordinate system and the reference spatial position. Based on the test results of the virtual-real superposition accuracy of the head-mounted display device under test corresponding to the dual cameras in the test pose, the test results of the virtual-real superposition accuracy of the head-mounted display device under test in the test pose are determined.

2. The method according to claim 1, characterized in that, The step of determining the virtual-real superposition accuracy test result of the head-mounted display device under test corresponding to the camera in the test pose, based on the second spatial position of the virtual marker in the virtual-real pair in the camera coordinate system and the reference spatial position, includes: If the distance between the second spatial position and the reference spatial position is less than a distance threshold, the test result of the virtual-real superposition accuracy of the head-mounted display device under test corresponding to the camera in the test pose is determined to be qualified. If the distance between the second spatial position and the reference spatial position is greater than or equal to the distance threshold, the test result of the virtual-real superposition accuracy of the head-mounted display device under test corresponding to the camera in the test pose is determined to be unqualified. And / or, determining the virtual-real superposition accuracy test result of the head-mounted display device under test in the test pose based on the virtual-real superposition accuracy test result of the head-mounted display device under test corresponding to the dual cameras in the test pose includes: If the test results of the virtual-real superposition accuracy of the head-mounted display device under test corresponding to the dual cameras are all qualified in the test pose, the test results of the virtual-real superposition accuracy of the head-mounted display device under test in the test pose are determined to be qualified. If the test result of the virtual-real superposition accuracy of the head-mounted display device under test corresponding to the dual cameras in the test pose is unqualified, the test result of the virtual-real superposition accuracy of the head-mounted display device under test in the test pose is determined to be unqualified.

3. The method according to claim 1, characterized in that, Before controlling the head-mounted display device under test to sequentially occupy different test poses, the method further includes: A second image is obtained by acquiring the image of the head-mounted display device under test and the calibration plate captured by the camera in a preset full-view pose; wherein, the second image includes the virtual-real pair; Based on the pixel coordinates of the real marker in the virtual-real pair in the second image, determine the real spatial position of the real marker in the camera coordinate system; Based on the pixel coordinates of the virtual marker in the second image, determine the virtual spatial position of the virtual marker in the camera coordinate system; The reference virtual-real relative pose is determined based on the real spatial position and the virtual spatial position.

4. The method according to claim 1, characterized in that, Before controlling the head-mounted display device under test to sequentially occupy different test poses, the method further includes: The head-mounted display device under test is controlled to move along a preset trajectory to initialize the pose detection system of the head-mounted display device under test.

5. The method according to claim 1, characterized in that, The first image pair obtained by acquiring the image of the head-mounted display device under test and the calibration plate captured by the dual cameras in the test pose includes: The first sub-image pair is obtained by acquiring images of the calibration board captured by the dual cameras in the test pose. Acquire the second sub-image pair obtained by the dual cameras capturing the image displayed by the head-mounted display device under test in the test pose; The first image pair is determined based on the first sub-image pair and the second sub-image pair.

6. The method according to claim 1, characterized in that, After determining the test results of the virtual-real superposition accuracy of the head-mounted display device under test, the method further includes: If the test result of the virtual-real overlay accuracy of the head-mounted display device under test is unqualified, the virtual-real overlay accuracy of the head-mounted display device under test shall be adjusted.

7. A virtual-real superposition accuracy testing system, comprising a robotic arm, dual cameras, a fixture, a calibration plate, a memory, and a processor, wherein the robotic arm is provided with the fixture at its end, the fixture being used to fix a head-mounted display device under test and the dual cameras, the memory being used to store computer instructions, and the processor being used to call the computer instructions from the memory to execute control the robotic arm to move the head-mounted display device under test and the dual cameras to different test poses in sequence, wherein the dual cameras are used to acquire the image displayed by the head-mounted display device under test in any test pose and the image of the calibration plate to obtain a first image pair, and the processor being used to call the computer instructions from the memory to execute the method as described in any one of claims 1 to 6.

8. The system according to claim 7, characterized in that, The robotic arm drives the head-mounted display device under test and the dual cameras to move sequentially around the center of the calibration plate in a plane parallel to the calibration plate by a preset angle, so that the head-mounted display device under test and the dual cameras are in different test poses in sequence; wherein, the preset angle is the ratio between 360° and the total number of test poses.

9. The system according to claim 7, characterized in that, The calibration board has a white circular mark at each of its four corners as a real mark, and the area of ​​the calibration board other than the real mark is black.

10. The system according to claim 7, characterized in that, The fixture is used to control the movement of at least one of the head-mounted display device under test and the dual cameras to adjust the relative position between the head-mounted display device under test and the dual cameras.

11. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images