Virtual-real superposition precision test method and system, and storage medium

Abstract

The invention discloses a virtual-real superposition precision test method and system and a storage medium, and relates to the technical field of wearable devices. The method comprises the following steps: controlling a to-be-tested head-mounted display device to be in different test poses in sequence; the method comprises the following steps of: acquiring a first image pair obtained by acquiring a picture displayed by a to-be-tested head-mounted display device under a test pose and an image of a calibration plate by double cameras, and obtaining a reference virtual-real relative pose according to a first spatial position actually marked in a camera coordinate system in a virtual-real pair of the first image and a reference virtual-real relative pose corresponding to the virtual-real pair; determining a reference space position of a virtual mark in the virtual-real pair under the camera coordinate system, and determining a virtual-real superposition precision test result of the to-be-tested head-mounted display device corresponding to the camera under the test pose according to a second space position of the virtual mark in the virtual-real pair under the camera coordinate system and the reference space position; and determining a virtual-real superposition precision test result of the to-be-tested head-mounted display device according to the virtual-real superposition precision test results of the to-be-tested head-mounted display device for the double cameras under different test poses.

Description

TECHNICAL FIELD

[0001] The present application relates to the wearable technology field, more particularly, to a virtual-real superposition accuracy testing method, a virtual-real superposition accuracy testing system, and a computer readable storage medium. BACKGROUND

[0002] When rendering virtual content, the head-mounted display device (AR / VR / MR) first reconstructs its pose in the real space through the camera-IMU, and then renders virtual content based on the pose. If the alignment error between the real coordinate system and the virtual coordinate system is too large, the virtual object will drift or misalign, so it is necessary to test the virtual-real superposition accuracy to ensure that the rendering position strictly corresponds to the real reference.

[0003] Currently, the existing virtual-real superposition accuracy testing methods mainly rely on the following two methods: (1) Subsystem unit testing: respectively calibrate and measure the error of independent components such as optical display module, IMU, and SLAM algorithm. Since the errors of each subsystem are coupled and superimposed after the integration of the whole machine, unit testing cannot reflect the virtual-real superposition accuracy error in the real use environment.

[0004] (2) Artificial subjective evaluation: the tester wears the device and observes the superposition accuracy of the virtual marker and the real marker by naked eye to give a score. This method is not only low in efficiency and poor in repeatability, but also cannot provide quantitative data, making it difficult to locate specific defect modules.

[0005] Therefore, the above two virtual-real superposition accuracy testing methods have obvious deficiencies in testing efficiency, quantitative accuracy, and systematic defect positioning. SUMMARY

[0006] An object of embodiments of the present application is to provide a new technical solution for virtual-real superposition accuracy testing to solve the problems of low testing efficiency and poor testing accuracy in the prior art when testing the virtual-real superposition accuracy of a head-mounted display device.

[0007] According to a first aspect of the present application, a virtual-real superposition accuracy testing method is provided, which comprises: controlling a to-be-tested head-mounted display device to be in different test poses in sequence; For any test pose, obtaining a first image pair of images of a display screen and a calibration board of the to-be-tested head-mounted display device in the test pose captured by double cameras, and determining a virtual-real superposition accuracy test result of the to-be-tested head-mounted display device in the test pose according to the first image pair; wherein the relative pose of the double cameras and the to-be-tested head-mounted display device is unchanged; determine the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-to-real overlay accuracy test results of the to-be-tested head-mounted display device in different test poses; wherein the first image in the first image pair includes at least one virtual-to-real pair, the virtual-to-real pair includes one real marker and one virtual marker, and the determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose according to the first image pair includes: for the first image corresponding to any camera in the first image pair, determining a reference spatial position of the virtual marker in the virtual-to-real pair in the camera coordinate system according to a first spatial position of the real marker in the virtual-to-real pair in the camera coordinate system and a reference virtual-to-real relative pose corresponding to the virtual-to-real pair, and determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera according to a second spatial position of the virtual marker in the virtual-to-real pair in the camera coordinate system and the reference spatial position; determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-to-real overlay accuracy test results of the to-be-tested head-mounted display device in the test pose corresponding to the dual cameras.

[0008] Optionally, the determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera according to the second spatial position of the virtual marker in the virtual-to-real pair in the camera coordinate system and the reference spatial position includes: in a case where a distance between the second spatial position and the reference spatial position is less than a distance threshold, determining that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera is qualified; in a case where the distance between the second spatial position and the reference spatial position is greater than or equal to the distance threshold, determining that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera is unqualified; and / or the determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-to-real overlay accuracy test results of the to-be-tested head-mounted display device in the test pose corresponding to the dual cameras includes: in a case where the virtual-to-real overlay accuracy test results of the to-be-tested head-mounted display device in the test pose corresponding to the dual cameras are all qualified, determining that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose is qualified; In a case where the to-be-tested head-mounted display device does not pass the virtual-real overlay accuracy test result corresponding to the dual cameras in the test pose, it is determined that the to-be-tested head-mounted display device does not pass the virtual-real overlay accuracy test result in the test pose.

[0009] Optionally, before the to-be-tested head-mounted display device is controlled to be in different test poses in sequence, the method further includes: obtaining a second image obtained by the camera capturing an image of the to-be-tested head-mounted display device displaying a picture in a preset all-view pose and the calibration board; wherein the second image includes the virtual-real pair; determining a real space position of the real marker in the camera coordinate system according to a pixel coordinate of the real marker in the virtual-real pair in the second image; determining a virtual space position of the virtual marker in the camera coordinate system according to a pixel coordinate of the virtual marker in the virtual-real pair in the second image; determining the reference virtual-real relative pose according to the real space position and the virtual space position.

[0010] Optionally, before the to-be-tested head-mounted display device is controlled to be in different test poses in sequence, the method further includes: controlling the to-be-tested head-mounted display device to move along a preset trajectory, so as to initialize a pose detection system of the to-be-tested head-mounted display device.

[0011] Optionally, the obtaining of the first image pair obtained by the dual cameras capturing an image of the to-be-tested head-mounted display device displaying a picture in the test pose and the calibration board includes: obtaining a first sub-image pair obtained by the dual cameras capturing an image of the calibration board in the test pose; obtaining a second sub-image pair obtained by the dual cameras capturing an image of the to-be-tested head-mounted display device displaying a picture in the test pose; determining the first image pair according to the first sub-image pair and the second sub-image pair.

[0012] Optionally, after the virtual-real overlay accuracy test result of the to-be-tested head-mounted display device is determined, the method further includes: in a case where the virtual-real overlay accuracy test result of the to-be-tested head-mounted display device is not qualified, performing virtual-real overlay accuracy adjustment processing on the to-be-tested head-mounted display device.

[0013] According to a second aspect of the present application, a virtual-real overlay precision test system is provided, comprising a mechanical arm, a dual camera, a clamp, a calibration board, a memory and a processor, the clamp is arranged at the end of the mechanical arm, the clamp is used for fixing a head-mounted display device to be tested and the dual camera, the memory is used for storing computer instructions, and the processor is used for calling the computer instructions from the memory to control the mechanical arm to drive the head-mounted display device to be tested and the dual camera to be in different test poses in sequence, the dual camera is used for collecting a first image pair of an image of the calibration board and a displayed picture of the head-mounted display device to be tested in any test pose, and the processor is used for calling the computer instructions from the memory to perform the method according to the first aspect.

[0014] Optionally, the mechanical arm drives the head-mounted display device to be tested and the dual camera to move around the center of the calibration board by a preset angle in parallel to the plane of the calibration board, so that the head-mounted display device to be tested and the dual camera are in different test poses in sequence; wherein the preset angle is a ratio between 360° and the total number of the test poses.

[0015] Optionally, the calibration board is provided with a white circular mark at each corner as a real mark, and the area of the calibration board other than the real mark is black.

[0016] Optionally, the clamp is used for controlling at least one of the head-mounted display device to be tested and the dual camera to move, so as to adjust the relative position between the head-mounted display device to be tested and the dual camera.

[0017] According to a third aspect of the present application, a computer readable storage medium is provided, and optionally, a computer program is stored on the computer readable storage medium, the computer program is executed by a processor to implement the method according to the first aspect.

[0018] An advantage of the present application is that the virtual-real overlay precision test in the test pose is automatically performed based on the first image pair obtained by the dual cameras collecting the image of the display screen of the to-be-tested head-mounted display device in the test pose and the image of the calibration board in the test pose. Since the first image pair is obtained according to the image of the display screen of the to-be-tested head-mounted display device in the test pose and the image of the calibration board in the test pose, it reflects the actual virtual-real overlay in the test pose, and thus the test based on the first image pair can avoid the defect that the unit test cannot reflect the virtual-real overlay error in the real use environment, improve the accuracy of the virtual-real overlay precision test, and avoid the problems of low test efficiency and low test accuracy caused by manual subjective evaluation, improve the accuracy and efficiency of the virtual-real overlay precision test. In addition, the virtual-real overlay precision of the to-be-tested head-mounted display device is evaluated based on the virtual-real overlay precision test results in different test poses, which can further improve the accuracy and efficiency of the test. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and, together with the description, serve to explain the principles of the application.

[0020] Figure 1 is a block diagram of a hardware configuration of a virtual-real overlay precision test system according to an embodiment of the present application; Figure 2 is a flowchart of a virtual-real overlay precision test method according to an embodiment of the present application; Figure 3 is a relative position between a virtual marker mapping point and a real marker of a to-be-tested head-mounted display device in a preset all-view pose according to an embodiment of the present application; Figure 4 is a relative position between a virtual marker mapping point and a real marker of a to-be-tested head-mounted display device in a test pose according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of components and steps set forth in these embodiments are not limiting to the scope of the present application unless otherwise specifically stated.

[0022] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way limiting to the scope of the application or its applications or uses.

[0023] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the specification, where appropriate.

[0024] In all of the examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as a limitation. Thus, other examples of the exemplary embodiments can have different values.

[0025] It should be noted that like numerals and letters refer to like items throughout the drawings, and once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0026] Figure 1 Fig. 1 is a block diagram of a hardware configuration of a virtual-real overlay precision test system 1000 according to an embodiment of the present application.

[0027] The virtual-real overlay precision test system 100 includes a mechanical arm 11, a dual camera 12, a clamp 13, a calibration board 14, a memory 15, and a processor 16.

[0028] The end of the mechanical arm 11 is provided with the clamp 13, which is used to fix the to-be-tested head-mounted display device and the dual camera 12. The memory 15 is used to store computer instructions, and the processor 16 is used to call the computer instructions from the memory 15 to execute control of the mechanical arm 11 to drive the to-be-tested head-mounted display device and the dual camera 12 to be in different test poses in sequence. The dual camera 12 is used to acquire a first image pair of an image of the calibration board 14 and a displayed picture of the to-be-tested head-mounted display device in any test pose. The processor 16 is used to call the computer instructions from the memory 15 to execute the virtual-real overlay precision test method of the present application.

[0029] In the present embodiment, the mechanical arm 11 adopts a multi-joint series design, and the movement of the mechanical arm 11 covers six degrees of freedom of pose changes to simulate multi-directional dynamic observation of the wearer's head.

[0030] The end of the mechanical arm 11 is integrated with the high-precision clamp 13, which is used to fix the to-be-tested head-mounted display device and the dual camera 12.

[0031] The to-be-tested head-mounted display device can be, for example, an AR device, an MR device, a VR device, etc., which is not limited here.

[0032] In an embodiment of the present application, the to-be-tested head-mounted display device is an MR device. Since the field of view of the MR device is larger and the display precision is higher, the dual camera can be replaced by two high-precision, large-view-angle RGB cameras.

[0033] The dual camera 12 is symmetrically arranged behind the display screen of the to-be-tested head-mounted display device to acquire the displayed picture of the to-be-tested head-mounted display device.

[0034] In an embodiment of the present application, to avoid the test error caused by misalignment of the optical axis of the camera and the optical center of the head-mounted display device when the dual-camera acquires the first image pair, the optical axis of the dual-camera 12 can be aligned with the optical center of the head-mounted display device to be tested.

[0035] During the whole test process, the relative pose of the dual-camera 12 and the head-mounted display device to be tested is unchanged.

[0036] In an embodiment of the present application, the fixture 13 is used to control the movement of at least one of the head-mounted display device to be tested and the dual-camera 12 to adjust the relative position between the head-mounted display device to be tested and the dual-camera 12.

[0037] In the present embodiment, to facilitate the alignment of the optical axis of the dual-camera 12 and the optical center of the head-mounted display device to be tested, at least one of the two fixtures used to fix the head-mounted display device to be tested and the dual-camera 12 can be a movable fixture. The relative position between the head-mounted display device to be tested and the dual-camera 12 is then adjusted by controlling the movement of the movable fixture.

[0038] The calibration plate 14 is arranged on one side of the mechanical arm 11 to facilitate the dual-camera 12 fixed on the mechanical arm 11 to acquire the image of the calibration plate.

[0039] The mechanical arm 11 drives the head-mounted display device to be tested and the dual-camera 12 to sequentially assume different test poses. Since the dual-camera 12 not only acquires the displayed image of the head-mounted display device to be tested in the test pose, but also acquires the image of the calibration plate 14 in the test pose, the first image pair is obtained. Therefore, the different test poses need to be determined based on the position of the calibration plate 14.

[0040] In an embodiment of the present application, the mechanical arm 11 drives the head-mounted display device to be tested and the dual-camera 12 to sequentially move around the center of the calibration plate 14 by a preset angle parallel to the plane of the calibration plate 14, so that the head-mounted display device to be tested and the dual-camera 12 sequentially assume different test poses.

[0041] In the present embodiment, the different test poses in combination need to meet the requirement of one round around the calibration plate 14. The center of the calibration plate 14 refers to the geometric center of the calibration plate 14. The preset angle is the ratio between 360° and the total number of test poses.

[0042] In an example, the total number of test poses is 10, and the preset angle is 36°.

[0043] In an embodiment of the present application, the four corners of the calibration plate 14 are respectively provided with a white circular mark as a real mark, and the area of the calibration plate 14 except the real mark is black.

[0044] In the present embodiment, as shown in FIG. 2, the four corners of the calibration plate 14 are respectively provided with a white circular mark as a real mark, and the area of the calibration plate 14 except the real mark is black. Figure 1As shown, the calibration board 14 has four large circles (i.e., four real markers) distributed at four corners, and other regions of the calibration board 14 are black. If the virtual markers generated by the head-mounted display device to be tested are projected into the three-dimensional space, the mapping points of the virtual markers in the three-dimensional space are distributed in the central black background region of the calibration board 14. In other words, the central black background region of the calibration board 14 is used to project the mapping points of the virtual markers generated by the head-mounted display device to be tested in the three-dimensional space (i.e., the mapping points of the virtual markers in the three-dimensional space are distributed in the central black background region of the calibration board 14). Figure 1 The small circles in the 4x5 array are the mapping points of the virtual markers in the three-dimensional space.

[0045] By using the anti-scattering black-and-white anti-color design calibration board, the scattering interference of the optical waveguide can be effectively suppressed.

[0046] The following describes the process of acquiring the first image pair by the dual cameras in one test pose, taking the head-mounted display device to be tested as an AR device as an example. In the case that the mechanical arm 11 fixes the AR device and the dual cameras 12, and the optical axis of the dual cameras 12 is aligned with the optical center of the AR device, the mechanical arm 11 drives the AR device and the dual cameras 12 to move 36° around the center of the calibration board 14 in a plane parallel to the calibration board 14, so that the AR device and the dual cameras 12 are in one test pose. In the test pose, the AR device will generate and display virtual markers based on the detected pose, and will also display real markers based on the real markers on the calibration board acquired by itself, that is, the screen displayed by the AR device at this time includes real markers and virtual markers. At this time, the dual cameras acquire the screen displayed by the AR device, and the first image pair is obtained. In another case, if the AR device only displays virtual markers without displaying real markers, the dual cameras can acquire real markers on the calibration board through the AR device while acquiring the screen displayed by the AR device, and the first image pair is obtained.

[0047] The following describes the process of acquiring the first image pair by the dual cameras in one test pose, taking the head-mounted display device to be tested as a VR device as an example. In the case that the mechanical arm 11 fixes the VR device and the dual cameras 12, and the optical axis of the dual cameras 12 is aligned with the optical center of the AR device, the mechanical arm 11 drives the VR device and the dual cameras 12 to move 36° around the center of the calibration board 14 in a plane parallel to the calibration board 14, so that the VR device and the dual cameras 12 are in one test pose. In the test pose, the VR device will generate and display virtual markers based on the detected pose, and the dual cameras acquire the screen displayed by the VR device to obtain the first sub-image pair. Then, the VR device is removed, and the mechanical arm 11 is controlled to make the dual cameras 12 in the test pose to obtain the second sub-image pair. Finally, the first sub-image pair and the second sub-image pair are spliced according to the corresponding cameras to splice the first sub-image and the second sub-image, and the first image of the corresponding camera is obtained, and then the first image pair is obtained.

[0048] The processor is configured to call the computer instructions from the memory to execute a virtual-real superimposition precision testing method provided by the embodiments of the present application to evaluate the virtual-real superimposition precision of the head-mounted display device to be tested.

[0049] When rendering virtual content, the head-mounted display device (AR / VR / MR) first reconstructs its pose in the real space through the camera-IMU, and then renders the virtual content based on the pose.

[0050] Ideally, after the head-mounted display device displays the generated virtual marker on the display screen, no matter how the pose of the head-mounted display device changes, the mapping point of the virtual marker in the three-dimensional space is fixed. For ease of description, the mapping point of the virtual marker in the three-dimensional space is referred to as the virtual marker mapping point. However, in the actual environment, the optical-IMU fusion positioning of the head-mounted display device has drift and noise, detection delay, lens distortion, calibration error, mechanical assembly tolerance, etc., which will cause the projection matrix of the virtual marker to deviate from the ideal value when generating virtual content, resulting in a shift in the position of the virtual marker mapping point. Therefore, the virtual-real superimposition precision of the head-mounted display device can be evaluated based on the degree of position shift of the virtual marker mapping point under multiple test poses.

[0051] The present application provides a virtual-real superimposition precision testing method, which is applied to a virtual-real superimposition precision testing system 100 as shown in Figure 1 The memory and the processor in the virtual-real superimposition precision testing system 100 as shown in Figure 2 The method comprises the following steps S2100 to S2300.

[0052] In step S2100, the head-mounted display device to be tested is controlled to be in different test poses in sequence.

[0053] In the present embodiment, the head-mounted display device to be tested can be an AR device, an MR device, a VR device, etc., which is not limited here.

[0054] The test pose can be a pose set by the tester according to the test requirements, and the specific position and direction of the test pose are not limited here.

[0055] In the virtual-real superimposition precision testing system as shown in Figure 1 In the virtual-real superimposition precision testing system as shown in

[0056] Since the dual camera not only needs to collect the display screen of the head-mounted display device to be tested in the test pose, but also needs to collect the image of the calibration board in the test pose, so as to obtain a first image pair. Therefore, the different test poses need to be determined based on the position of the calibration board.

[0057] In an embodiment of the present application, the step S2100 of controlling the to-be-tested head-mounted display device to sequentially be in different test poses comprises: controlling the mechanical arm to drive the to-be-tested head-mounted display device and the dual camera to sequentially move a preset angle around the center of the calibration plate in a plane parallel to the calibration plate, so that the to-be-tested head-mounted display device and the dual camera sequentially are in different test poses.

[0058] In the embodiment, the different test poses in combination meet the requirement of going around the calibration plate once. The center of the calibration plate refers to the geometric center of the calibration plate. The preset angle is a ratio between 360° and the total number of test poses.

[0059] In an example, the total number of test poses is 10, and the preset angle is 36°. In an embodiment of the present application, before the step S2100 of controlling the to-be-tested head-mounted display device to sequentially be in different test poses, the method further comprises: a step S2000.

[0060] The step S2000 controls the to-be-tested head-mounted display device to move along a preset trajectory, so that a pose detection system of the to-be-tested head-mounted display device is initialized.

[0061] In the embodiment, the preset trajectory can be a trajectory designed for the pose detection system initialization of the to-be-tested head-mounted display device, and the specific shape of the preset trajectory is not limited here.

[0062] The pose detection system of the to-be-tested head-mounted display device comprises at least one of an IMU (Inertial Measurement Unit) and a SLAM (Simultaneous Localization and Mapping), and the pose detection system initialization is to make the pose detection system converge, thereby improving the pose control accuracy of the to-be-tested head-mounted display device.

[0063] The step S2200, for any test pose, acquires a first image pair obtained by the dual camera collecting images of a display picture of the to-be-tested head-mounted display device in the test pose and the calibration plate, and determines a virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose according to the first image pair.

[0064] In the step S2200, the relative pose between the dual camera and the to-be-tested head-mounted display device is unchanged.

[0065] For example, the head-mounted display device to be tested is an AR device: in any test pose, the AR device generates and displays virtual markers based on the detected pose of the AR device, and displays real markers based on the real markers on the calibration board captured by the AR device, that is, the AR device displays a picture including real markers and virtual markers. At this time, the dual cameras capture the picture displayed by the AR device to obtain a first image pair. In another case, if the AR device only displays virtual markers and does not display real markers, the dual cameras can capture real markers on the calibration board through the AR device while capturing the picture displayed by the AR device, thereby obtaining a first image pair.

[0066] In an embodiment of the present application, the head-mounted display device to be tested is a VR / MR device.

[0067] When the user wears the VR / MR device, the user cannot see the external real world, that is, the dual cameras cannot capture real markers on the calibration board and the picture displayed by the VR / MR device at the same time in a test pose.

[0068] Therefore, in an embodiment of the present application, the first image pair obtained by capturing the picture displayed by the head-mounted display device to be tested and the image of the calibration board in the test pose by the dual cameras in step S2200 includes steps S2200.1-S2200.3.

[0069] In step S2200.1, a first sub-image pair captured by the dual cameras in the test pose is obtained.

[0070] In this embodiment, when the mechanical arm does not carry the VR / MR device, the mechanical arm is controlled to reach a test pose, and the dual cameras capture the calibration board to obtain a first sub-image pair. The first sub-image pair is an image of the calibration board in the view angle corresponding to the test pose.

[0071] In step S2200.2, a second sub-image pair captured by the dual cameras in the test pose is obtained.

[0072] In this embodiment, when the mechanical arm carries the VR / MR device, the mechanical arm is controlled to drive the VR / MR device to reach the test pose, and the dual cameras capture the picture displayed by the VR / MR device at this time to obtain a second sub-image pair. The second sub-image pair is an image of the virtual markers displayed by the VR / MR device in the test pose.

[0073] In step S2200.3, the first image pair is determined according to the first sub-image pair and the second sub-image pair.

[0074] In this embodiment, the first sub-image and the second sub-image corresponding to each camera are merged to obtain a first image corresponding to the camera. Then, a first image pair corresponding to the two cameras can be obtained.

[0075] In the example where the calibration board includes 4 real markers and a 4x5 array of virtual marker mapping points, the first image obtained by any camera of the two cameras capturing the image of the display screen of the head-mounted display device under test displayed at the test pose and the calibration board under the test pose can include only one real marker and one virtual marker, can include 4 real markers and a 4x5 array of virtual markers (i.e. 20 real-virtual pairs), or can include 1 real marker and 2 virtual markers (i.e. two real-virtual pairs), which is not limited here.

[0076] That is, the first image includes at least one real-virtual pair. The real-virtual pair includes one real marker and one virtual marker.

[0077] In this embodiment, the step S2200 of determining the real-virtual superimposition accuracy test result of the head-mounted display device under test at the test pose according to the first image pair includes steps S3100-S3200.

[0078] In step S3100, for the first image corresponding to any camera in the first image pair, the reference spatial position of the virtual marker in the real-virtual pair in the camera coordinate system is determined according to the first spatial position of the real marker in the real-virtual pair in the camera coordinate system and the reference real-virtual relative pose corresponding to the real-virtual pair, and the real-virtual superimposition accuracy test result of the head-mounted display device under test corresponding to the camera at the test pose is determined according to the second spatial position of the virtual marker in the real-virtual pair in the camera coordinate system and the reference spatial position.

[0079] In this embodiment, for each camera in the two cameras, the camera coordinate system of the camera is a coordinate system established with the optical axis of the camera as the origin. Since the head-mounted display device under test and the two cameras are carried on the robot arm, the positions of the two cameras in the three-dimensional space also change accordingly during the movement of the robot arm driving the head-mounted display device under test, and thus the camera coordinate systems corresponding to the two cameras also change, and the three-dimensional spatial positions of the real markers and the virtual markers in the camera coordinate systems also change. Therefore, each time the head-mounted display device under test is moved, the first spatial position of the real marker in the camera coordinate system needs to be calculated based on the pixel coordinates of the real marker in the first image captured by any camera under the current test pose and the back-projection parameters of the camera, and the second spatial position of the virtual marker in the camera coordinate system needs to be calculated based on the pixel coordinates of the virtual marker in the first image and the back-projection parameters of the camera.

[0080] Based on this, in one embodiment of the present application, before step S3100 is performed, the method further comprises step SA.

[0081] In step SA, for a first image corresponding to any camera in the first image pair, a first spatial position of the real marker in the camera coordinate system is determined according to the pixel coordinates of the real marker in the virtual-real pair of the first image and the back-projection parameters of the camera, and a second spatial position of the virtual marker in the camera coordinate system is determined according to the pixel coordinates of the virtual marker in the virtual-real pair of the first image and the back-projection parameters of the camera.

[0082] In the present embodiment, the back-projection parameters can include the extrinsic parameters and the intrinsic parameters of the camera.

[0083] The back-projection parameters can be determined according to the properties of the camera, or can be determined after the calibration of the back-projection parameters of the camera before the present virtual-real superimposition accuracy test, which is not limited here.

[0084] In one embodiment of the present application, before step S2100 controls the to-be-tested head-mounted display device to be in different test poses in turn, the method further comprises steps SA1-S A2. That is, the back-projection parameters of the camera in step SA are determined through steps SA1-S A2.

[0085] In step SA1, a set of calibration images of the calibration board acquired by the camera in different calibration poses is obtained.

[0086] In the present embodiment, in the case where the mechanical arm is not fixed to the to-be-tested head-mounted display device, the mechanical arm is controlled to drive the camera to move to each of the different calibration poses in turn to acquire the calibration images of the calibration board in the view angle corresponding to the calibration pose, thereby obtaining the set of calibration images. The different calibration poses can be the same as or different from the different test poses, which is not limited here.

[0087] In step SA2, the back-projection parameters of the camera are determined according to the set of calibration images.

[0088] In the present embodiment, corner point detection is performed on each calibration image in the set of calibration images, and the extrinsic parameters and the intrinsic parameters of the camera are determined based on the detected corner points as the back-projection parameters of the camera.

[0089] The relative pose between the virtual and real objects is the ideal relative spatial position relationship between the virtual-real pair in the camera coordinate system.

[0090] For the convenience of description, the large circle at the upper left corner of the calibration board in Figure 1 is referred to as the first real marker, and the large circle at the upper right corner of the calibration board in Figure 1The smallest circle closest to the left upper corner in the 4x5 array (i.e., the smallest circle in the first row and the first column in the 4x5 array) is referred to as the first virtual marker, and the two constitute a virtual-real pair. Taking the virtual-real pair as an example, the virtual-real superposition accuracy test method of the present application is described. That is, for the virtual-real pair, there is a reference virtual-real relative pose corresponding to the left camera and a reference virtual-real relative pose corresponding to the right camera. The reference virtual-real relative pose reflects the ideal relative spatial position relationship of the virtual-real pair in the corresponding camera coordinate system.

[0091] The reference virtual-real relative pose can be obtained according to the relative spatial position relationship of the virtual-real pair in the corresponding camera coordinate system when the head-mounted display device is in an ideal state (i.e., ideal pose detection accuracy and no detection delay, etc.).

[0092] In an embodiment of the present application, before step S2100 controls the to-be-tested head-mounted display device to be in different test poses in turn, the method further comprises steps S4100-S4400.

[0093] In step S4100, a second image obtained by the camera capturing an image of the to-be-tested head-mounted display device displaying a picture in a preset full-view pose and the calibration board is acquired.

[0094] The second image includes the virtual-real pair.

[0095] In the present embodiment, the preset full-view pose can be a pose that can make the real markers on the calibration board completely fall within the field of view of the camera.

[0096] The camera in the present step and the camera in step S3100 are corresponding, that is, if the camera in step S3100 is a left camera, then the camera in the present step is a left camera. If the camera in step S3100 is a right camera, then the camera in the present step is a right camera.

[0097] In one example, the second image includes a virtual-real pair composed of the first real marker 1 and the first virtual marker 2.

[0098] In step S4200, the real spatial position of the real marker in the camera coordinate system is determined according to the pixel coordinates of the real marker in the virtual-real pair.

[0099] In the present embodiment, the real spatial position of the real marker in the camera coordinate system is determined based on the pixel coordinates of the real marker in the virtual-real pair and the back-projection parameters of the camera.

[0100] Continuing the above example, in the example where the second image includes a virtual-real pair composed of the first real marker 1 and the first virtual marker 2, the real spatial position of the first real marker 1 in the camera coordinate system can be as shown in Figure 3 ​

[0101] Step S4300: Determine the virtual spatial position of the virtual marker in the camera coordinate system based on the pixel coordinates of the virtual marker in the virtual-real alignment.

[0102] In this embodiment, the virtual spatial position of the virtual marker in the camera coordinate system is determined based on the pixel coordinates of the virtual marker in the virtual-real alignment and the back projection parameters of the camera. The virtual spatial position is the spatial location of the virtual marker's spatial mapping point in the camera coordinate system.

[0103] Continuing with the example above, in the example where the second image includes a real-virtual pair consisting of a first real marker 1 and a first virtual marker 2, the virtual spatial position of the first virtual marker 2 in the camera coordinate system can be as follows: Figure 3 As shown.

[0104] Step S4400: Determine the reference virtual-real relative pose of the virtual-real pair based on the real spatial position and the virtual spatial position.

[0105] In this embodiment, the reference virtual-to-real relative pose can be a reference rotation matrix and a reference translation matrix.

[0106] Continuing with the example above, based on Figure 3 The relative positions of the first real marker 1 and the first virtual marker 2 are used to determine the reference virtual-real relative pose of the virtual-real pair.

[0107] Regardless of the movement of the robotic arm, the reference relative pose of the virtual-real pair remains constant. In other words, regardless of the movement of the robotic arm, the relative positional relationship between the mapping points of the real and virtual markers in the virtual-real pair on the calibration board remains unchanged. Therefore, based on the first spatial position of the real marker in the camera coordinate system and the reference relative pose of the virtual-real pair, the ideal spatial position of the virtual marker in the camera coordinate system, i.e., the reference spatial position, can be obtained.

[0108] For example, such as Figure 4 As shown, in the test pose, the first real marker 1 has a first spatial position in the camera coordinate system at the position shown, and the first virtual marker 2 has a second spatial position in the camera coordinate system at the position shown (i.e., the actual spatial position of the first virtual marker). Based on the first spatial position of the first real marker 1 in the camera coordinate system and the reference virtual-real relative pose corresponding to this virtual-real pair, the reference spatial position of the first virtual marker in the camera coordinate system is determined, that is, the ideal spatial position of the first virtual marker, as shown. Figure 4 The position corresponding to mark 3 in the middle.

[0109] In one embodiment of this application, the reference relative pose includes a reference rotation matrix and a reference translation matrix.

[0110] In this embodiment, the step S3100 of determining, according to the first spatial position of the virtual marker in the camera coordinate system and the reference rotation matrix and the reference translation matrix, the reference spatial position of the virtual marker in the camera coordinate system comprises the following steps: According to the first spatial position and the reference rotation matrix and the reference translation matrix, the reference spatial position of the virtual marker in the camera coordinate system is determined.

[0111] In this embodiment, the reference rotation matrix is used for rotation and the reference translation matrix is used for translation of the first spatial position, so that the reference spatial position of the virtual marker in the camera coordinate system is obtained.

[0112] Based on the second spatial position and the reference spatial position of the virtual marker in the camera coordinate system, a virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device corresponding to the camera in the test pose is determined. The virtual-to-real superimposition accuracy test result is qualified or unqualified.

[0113] In an embodiment of the present application, the step S3100 of determining, according to the second spatial position and the reference spatial position of the virtual marker in the camera coordinate system, the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device corresponding to the camera in the test pose comprises the following steps S3100.1 and S3100.2.

[0114] The step S3100.1 comprises the following step: in a case where the distance between the second spatial position and the reference spatial position is less than a distance threshold, determining that the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device corresponding to the camera in the test pose is qualified.

[0115] In this embodiment, the distance threshold can be flexibly set according to the requirement of test accuracy, which is not limited herein.

[0116] The step S3100.2 comprises the following step: in a case where the distance between the second spatial position and the reference spatial position is greater than or equal to the distance threshold, determining that the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device corresponding to the camera in the test pose is unqualified.

[0117] The step S3200 comprises the following step: determining the virtual-to-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-to-real superimposition accuracy test results of the to-be-tested head-mounted display device corresponding to the dual cameras in the test pose.

[0118] In an embodiment of the present application, step S3200 determines the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-to-real overlay accuracy test results of the to-be-tested head-mounted display device corresponding to the dual cameras in the test pose, including step S3200.1 and step S3200.2.

[0119] Step S3200.1, in the case where the virtual-to-real overlay accuracy test results of the to-be-tested head-mounted display device corresponding to the dual cameras in the test pose are all qualified, determining that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose is qualified.

[0120] Step S3200.2, in the case where the virtual-to-real overlay accuracy test results of the to-be-tested head-mounted display device corresponding to the dual cameras in the test pose exist unqualified, determining that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device in the test pose is unqualified.

[0121] Since the motion of a human head is complex and changeable, it is not limited to the virtual-to-real overlay accuracy test in one test pose, and therefore, in order to improve the accuracy of the virtual-to-real overlay accuracy test, the virtual-to-real overlay accuracy test can be performed in multiple test poses to accurately evaluate the pose detection accuracy of the to-be-tested head-mounted display device.

[0122] Step S2300, determining the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device according to the virtual-to-real overlay accuracy test results of the to-be-tested head-mounted display device in the different test poses. In the embodiment, in the case where the virtual-to-real overlay accuracy test results of the to-be-tested head-mounted display device in different test poses are all qualified, it is determined that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device is qualified. In the case where the virtual-to-real overlay accuracy test results of the to-be-tested head-mounted display device in different test poses exist unqualified, it is determined that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device is unqualified. For example, 10 test poses are set, for each test pose, step S2200 is performed to obtain 10 virtual-to-real overlay accuracy test results corresponding to 10 test poses. In the case where the 10 virtual-to-real overlay accuracy test results are all qualified, it is determined that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device is qualified. If there is unqualified in the 10 virtual-to-real overlay accuracy test results, it is determined that the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device is unqualified.

[0123] In an embodiment of the present application, after step S2300 determines the virtual-to-real overlay accuracy test result of the to-be-tested head-mounted display device, the method further includes: In a case where the virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device is unqualified, the virtual-real superimposition accuracy adjustment processing is performed on the to-be-tested head-mounted display device.

[0124] In this embodiment, the virtual-real superimposition accuracy adjustment processing can be detecting the drift of the pose detection system of the to-be-tested head-mounted display device, performing optical re-calibration on the to-be-tested head-mounted display device, and detecting the perpendicularity error of the to-be-tested head-mounted display device, so as to improve the virtual-real superimposition accuracy of the to-be-tested head-mounted display device.

[0125] The first image pair obtained by capturing the image of the display screen of the to-be-tested head-mounted display device and the image of the calibration board in the test pose by the dual cameras is used to automatically perform the virtual-real superimposition accuracy test in the test pose. Since the first image pair is obtained according to the image of the display screen of the to-be-tested head-mounted display device and the image of the calibration board in the test pose, it reflects the actual virtual-real superimposition in the test pose. Therefore, the test based on the first image pair can avoid the defect that the unit test cannot reflect the virtual-real superimposition error in the real use environment, improve the accuracy of the virtual-real superimposition accuracy test, and avoid the problems of low test efficiency and low test accuracy caused by manual subjective evaluation, thereby improving the accuracy and efficiency of the virtual-real superimposition accuracy test. In addition, the virtual-real superimposition accuracy of the to-be-tested head-mounted display device is evaluated based on the virtual-real superimposition accuracy test results in different test poses, which can further improve the accuracy and efficiency of the test.

[0126] The virtual-real superimposition accuracy test method of one embodiment of the present application will be described in detail below by taking a to-be-tested head-mounted display device as an example, which includes steps S1-S10.

[0127] In step S1, the dual cameras and the to-be-tested AR device are installed on the clamp of the mechanical arm, and the optical axis of the dual cameras and the optical center of the to-be-tested AR device are aligned.

[0128] In step S2, the to-be-tested AR device is removed from the clamp, and the mechanical arm is controlled to move to different calibration poses in sequence to obtain a calibration image set obtained by capturing the image of the calibration board by the camera in different calibration poses.

[0129] In this example, the calibration board includes four real markers.

[0130] In step S3, the back projection parameters of the camera are determined according to the calibration image set.

[0131] In this example, the back projection parameters include camera extrinsic parameters and camera intrinsic parameters.

[0132] In step S4, the to-be-tested AR device is placed on the clamp, and the to-be-tested AR device is controlled to move along a preset trajectory to initialize the pose detection system of the to-be-tested AR device.

[0133] Step S5, the control unloading arm drives the AR device to be tested to move to a preset all-view pose, and a second image is acquired by the camera.

[0134] The second image includes 4 real markers and a 4x5 array of virtual markers, i.e., there are 20 real-virtual pairs in the second image.

[0135] Step S6, for any real-virtual pair of the 20 real-virtual pairs, a reference real-virtual relative pose corresponding to the real-virtual pair is determined, and 20 reference real-virtual relative poses corresponding to the 20 real-virtual pairs are obtained.

[0136] Specifically, the reference real-virtual relative pose corresponding to a real-virtual pair is determined as follows: first, the real space position of the real marker in the camera coordinate system is determined according to the pixel coordinates of the real marker in the real-virtual pair and the back projection parameters of the camera. Then, the virtual space position of the virtual marker in the camera coordinate system is determined according to the pixel coordinates of the virtual marker in the real-virtual pair and the back projection parameters of the camera. The reference real-virtual relative pose of the real-virtual pair is determined according to the real space position and the virtual space position.

[0137] Step S7, the control mechanical arm drives the AR device to be tested and the dual camera to move 36° around the center of the calibration board in a plane parallel to the calibration board, so that the AR device to be tested and the dual camera are sequentially in 10 test poses.

[0138] Step S8, for any test pose, a first image pair of the images of the display screen of the AR device to be tested and the calibration board in the test pose acquired by the dual camera is obtained, and a real-virtual superimposition accuracy test result of the AR device to be tested in the test pose is determined according to the first image pair.

[0139] The first image in the first image pair includes 4 real-virtual pairs, and each real-virtual pair includes one real marker and one virtual marker. The following is an example to illustrate how to determine the real-virtual superimposition accuracy test result of the AR device to be tested in a test pose according to the first image pair in the test pose: For any one of the virtual-real pairs in any one of the first images corresponding to any one pair of cameras in the first image pair, a reference spatial position of the virtual marker in the virtual-real pair in the camera coordinate system is determined according to a first spatial position of the real marker in the virtual-real pair in the camera coordinate system and a reference virtual-real relative pose corresponding to the virtual-real pair. A virtual-real superimposition accuracy test result corresponding to the virtual-real pair is determined according to a second spatial position of the virtual marker in the virtual-real pair in the camera coordinate system and the reference spatial position. A virtual-real superimposition accuracy test result of the AR device under test corresponding to the camera at the test pose is determined according to four virtual-real superimposition accuracy test results corresponding to four virtual-real pairs in the first image. A virtual-real superimposition accuracy test result of the AR device under test at the test pose is determined according to virtual-real superimposition accuracy test results of the AR device under test corresponding to the dual cameras at the test pose.

[0140] The determining of the virtual-real superimposition accuracy test result corresponding to the virtual-real pair according to the second spatial position of the virtual marker in the virtual-real pair in the camera coordinate system and the reference spatial position includes: In a case where a distance between the second spatial position and the reference spatial position is less than a distance threshold, it is determined that the virtual-real superimposition accuracy test result corresponding to the virtual-real pair is qualified. In a case where the distance between the second spatial position and the reference spatial position is greater than or equal to the distance threshold, it is determined that the virtual-real superimposition accuracy test result corresponding to the virtual-real pair is unqualified.

[0141] The determining of the virtual-real superimposition accuracy test result of the AR device under test corresponding to the camera at the test pose according to the four virtual-real superimposition accuracy test results corresponding to the four virtual-real pairs in the first image includes: In a case where the four virtual-real superimposition accuracy test results corresponding to the four virtual-real pairs are all qualified, it is determined that the virtual-real superimposition accuracy test result of the AR device under test corresponding to the camera at the test pose is qualified. In a case where the four virtual-real superimposition accuracy test results corresponding to the four virtual-real pairs are unqualified, it is determined that the virtual-real superimposition accuracy test result of the AR device under test corresponding to the camera at the test pose is unqualified.

[0142] The determining of the virtual-real superimposition accuracy test result of the AR device under test at the test pose according to the virtual-real superimposition accuracy test results of the AR device under test corresponding to the dual cameras at the test pose includes: In a case where the virtual-real superimposition accuracy test results of the AR device under test corresponding to the dual cameras at the test pose are all qualified, it is determined that the virtual-real superimposition accuracy test result of the AR device under test at the test pose is qualified. In a case where the virtual-real superimposition accuracy test results of the AR device under test corresponding to the dual cameras at the test pose are unqualified, it is determined that the virtual-real superimposition accuracy test result of the AR device under test at the test pose is unqualified.

[0143] Step S9, in the case that the 10 virtual-real superimposition accuracy test results corresponding to the 10 test poses are all qualified, determining that the virtual-real superimposition accuracy test result of the AR device under test is qualified.

[0144] Step S10, in the case that the 10 virtual-real superimposition accuracy test results corresponding to the 10 test poses are not qualified, determining that the virtual-real superimposition accuracy test result of the AR device under test is unqualified.

[0145] The application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the method described in any of the method embodiments when executed by a processor.

[0146] The present application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0147] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched-tape, a holographic storage medium, or any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0148] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0149] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0150] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0151] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or nonvolatile memory, or a suitable combination of the different types of computer readable storage media. The computer readable program instructions can also be downloaded to a computer, other programmable data processing apparatus, or other device from a computer readable storage medium or to an external computer or external storage device via a data signal that can be transmitted for example via a wired medium or a wireless medium such as the Internet or Wireless Application Protocol (WAP) signaling.

[0152] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0153] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0154] Embodiments of the application have been described above. The descriptions are intended to be illustrative, and not restrictive, of possible embodiments of the application. Many modifications and variations of the described embodiments are possible, given the benefit of the present disclosure, without departing from the scope and spirit of the described embodiments. The scope of the application is defined by the appended claims.

Claims

1. A virtual-to-real overlay accuracy testing method, characterized in that, The method comprises: controlling a to-be-tested head-mounted display device to be in different test poses in turn; for any test pose, obtaining a first image pair of images of a display screen of the to-be-tested head-mounted display device and a calibration board in the test pose captured by double cameras, and determining a virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose according to the first image pair, wherein a relative pose of the double cameras and the to-be-tested head-mounted display device is unchanged; determining a virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device according to virtual-real superimposition accuracy test results of the to-be-tested head-mounted display device in different test poses; and wherein a first image in the first image pair comprises at least one virtual-real pair, the virtual-real pair comprises one real marker and one virtual marker, and the determining of the virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose according to the first image pair comprises: for any first image corresponding to a camera in the first image pair, determining a reference spatial position of a virtual marker in the virtual-real pair in a camera coordinate system according to a first spatial position of a real marker in the virtual-real pair in the camera coordinate system and a reference virtual-real relative pose corresponding to the virtual-real pair, and determining a virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera according to a second spatial position of the virtual marker in the virtual-real pair in the camera coordinate system and the reference spatial position; determining a virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose according to virtual-real superimposition accuracy test results of the to-be-tested head-mounted display device in the test pose corresponding to the double cameras.

2. The method of claim 1, wherein, The determining of the virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera according to the second spatial position of the virtual marker in the virtual-real pair in the camera coordinate system and the reference spatial position comprises: in a case where a distance between the second spatial position and the reference spatial position is less than a distance threshold, determining that the virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera is qualified; in a case where the distance between the second spatial position and the reference spatial position is greater than or equal to the distance threshold, determining that the virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose corresponding to the camera is unqualified; and / or the determining of the virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose according to the virtual-real superimposition accuracy test results of the to-be-tested head-mounted display device in the test pose corresponding to the double cameras comprises: in a case where the virtual-real superimposition accuracy test results of the to-be-tested head-mounted display device in the test pose corresponding to the double cameras are all qualified, determining that the virtual-real superimposition accuracy test result of the to-be-tested head-mounted display device in the test pose is qualified. In a case where the to-be-tested head-mounted display device does not pass the virtual-real overlay accuracy test result corresponding to the dual cameras in the test pose, it is determined that the to-be-tested head-mounted display device does not pass the virtual-real overlay accuracy test result in the test pose.

3. The method of claim 1, wherein, Before the to-be-tested head-mounted display device is controlled to be in different test poses in sequence, the method further comprises: obtaining a second image obtained by the camera collecting an image of the to-be-tested head-mounted display device displaying a picture in a preset all-view pose and the calibration board, wherein the second image comprises the virtual-real pair; determining a real space position of the real marker in the camera coordinate system according to a pixel coordinate of the real marker in the virtual-real pair in the second image; determining a virtual space position of the virtual marker in the camera coordinate system according to a pixel coordinate of the virtual marker in the virtual-real pair in the second image; determining the reference virtual-real relative pose according to the real space position and the virtual space position.

4. The method of claim 1, wherein, Before the to-be-tested head-mounted display device is controlled to be in different test poses in sequence, the method further comprises: controlling the to-be-tested head-mounted display device to move along a preset trajectory to initialize a pose detection system of the to-be-tested head-mounted display device.

5. The method of claim 1, wherein, The obtaining of the first image pair obtained by the dual cameras collecting images of the to-be-tested head-mounted display device displaying a picture in the test pose and the calibration board comprises: obtaining a first sub-image pair obtained by the dual cameras collecting images of the calibration board in the test pose; obtaining a second sub-image pair obtained by the dual cameras collecting images of the to-be-tested head-mounted display device displaying a picture in the test pose; determining the first image pair according to the first sub-image pair and the second sub-image pair.

6. The method of claim 1, wherein, After determining the virtual-real overlay accuracy test result of the to-be-tested head-mounted display device, the method further comprises: in a case where the virtual-real overlay accuracy test result of the to-be-tested head-mounted display device is not qualified, performing virtual-real overlay accuracy adjustment processing on the to-be-tested head-mounted display device.

7. A virtual-real overlay accuracy test system, comprising a mechanical arm, dual cameras, a clamp, a calibration board, a memory and a processor, an end of the mechanical arm is provided with the clamp, the clamp is used for fixing a to-be-tested head-mounted display device and the dual cameras, the memory is used for storing computer instructions, and the processor is used for calling the computer instructions from the memory to perform control on the mechanical arm to drive the to-be-tested head-mounted display device and the dual cameras to be in different test poses in sequence, the dual cameras are used for collecting a first image pair obtained by collecting images of the to-be-tested head-mounted display device displaying a picture in any test pose and the calibration board, and the processor is used for calling the computer instructions from the memory to perform the method in any one of claims 1 to 6.

8. The system of claim 7, wherein, The mechanical arm drives the to-be-tested head-mounted display device and the dual camera to move in sequence around the center of the calibration board by a preset angle in a plane parallel to the calibration board, so that the to-be-tested head-mounted display device and the dual camera are in different test poses in sequence; wherein the preset angle is a ratio between 360° and the total number of the test poses.

9. The system of claim 7, wherein, Four corners of the calibration board are respectively provided with a white circular mark as a real mark, and an area of the calibration board except the real mark is black.

10. The system of claim 7, wherein, The clamp is used to control movement of at least one of the to-be-tested head-mounted display device and the dual camera, so as to adjust a relative position between the to-be-tested head-mounted display device and the dual camera.

11. A computer readable storage medium, characterized in that, The computer program is stored thereon and is used to implement the method according to any one of claims 1-6 when executed by the processor.

Images