Test method, test device and test system of augmented reality equipment, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a test method, a test device and a test system of augmented reality equipment, electronic equipment, a storage medium and a program product. The first camera is located at the eye spot position of the augmented reality device; the method comprises the steps that a virtual-real fusion picture which is shot by a first camera frame by frame and displayed by the augmented reality device is obtained, the virtual-real fusion picture is a picture obtained after a virtual picture and a reality picture of the augmented reality device are overlapped, the virtual picture comprises a virtual target, and the reality picture comprises an image which is shot by the augmented reality device and contains the reality target; obtaining trajectory data of the virtual target and the real target based on the virtual-real fusion picture; and according to the trajectory data of the virtual target and the real target, obtaining an evaluation index of the augmented reality device, the evaluation index being used for evaluating the fusion degree of the virtual picture and the real picture of the augmented reality device. The method is used for achieving the effect that the monitoring of the virtual-real fusion degree is more accurate.

Description

Technical Field

[0001] This application relates to the field of screen testing technology, and in particular to a testing method, testing device, testing system, electronic device, storage medium and program product for extended reality devices. Background Technology

[0002] Extended Reality (XR) is a technology that creates virtual environments and allows users to interact with real-world scenes. For example, XR devices can use video-see-through (VST) to capture real-world images reflecting the real scene, and then render and output virtual images reflecting the virtual environment. This virtual image is superimposed on the real-world image, forming a blended virtual-real image projected onto the XR device's display screen. Users perceive the scene through this blended virtual-real image on the display screen.

[0003] XR devices can also use optical see-through (OST) technology to capture realistic images that reflect the real scene. When using optical see-through to capture realistic images, the real scene can be directly seen through the transparent optical components at the front of the XR device, thus capturing realistic images that reflect the real scene.

[0004] Since real-world scenes and virtual environments come from different links, an important factor affecting user immersion during the actual experience is the degree of integration between the virtual and real images, that is, whether the constructed virtual environment can match the real scene in real time and provide users with a better extended reality experience.

[0005] However, given that XR devices are terminal integration devices, the process of acquiring real-world scenes and constructing virtual environments is constrained by various factors such as hardware, algorithms, and usage scenarios. Therefore, finding a better testing method and metric to evaluate the degree of virtual-real image fusion in assembled XR smart devices is an urgent problem to be solved. Summary of the Invention

[0006] This application provides a testing method, testing device, testing system, electronic device, storage medium, and program product for extended reality devices. These methods aim to achieve more accurate evaluation of the fusion degree between virtual and real images by recording the changes of virtual and real targets frame by frame in real time, thereby reducing evaluation errors caused by hardware precision limitations and making the monitoring of the virtual-real fusion degree more precise.

[0007] In a first aspect, embodiments of this application provide a testing method for an extended reality device, wherein a first camera is located at the eye point of the extended reality device; the method includes: acquiring frame-by-frame images of a virtual-real fusion display on the extended reality device captured by the first camera, the virtual-real fusion image being an overlay of a virtual image and a real image of the extended reality device, the virtual image including a virtual target, and the real image including an image captured by the extended reality device containing a real target; acquiring trajectory data of the virtual target and the real target based on the virtual-real fusion image; and obtaining an evaluation index for the extended reality device based on the trajectory data of the virtual target and the real target, the evaluation index being used to evaluate the degree of fusion between the virtual image and the real image of the extended reality device.

[0008] In one possible implementation, the virtual-real fusion image includes: a static virtual-real fusion image displayed on a stationary extended reality device, captured frame by frame by the first camera; and / or a dynamic virtual-real fusion image displayed on a moving extended reality device, captured frame by frame by the first camera.

[0009] In one possible implementation, the virtual fusion image includes a static virtual-real fusion image, the trajectory data is static trajectory data, and the evaluation index includes static jitter rate and / or relative jitter error; the static jitter rate index represents the proportion of virtual target jitter and real target jitter greater than a preset first jitter threshold between different frames in the static state; the relative jitter error index represents the relative change of virtual target and real target between different frames in the static state.

[0010] In one possible implementation, the virtual fusion image includes a dynamic virtual-real fusion image, the trajectory data is dynamic trajectory data, and the evaluation index includes a dynamic virtual-real stuttering rate index and / or a dynamic drift index; the dynamic virtual-real stuttering rate index represents the proportion of virtual target jitter and real target jitter less than a preset second jitter threshold between different frames in the motion state; the dynamic drift index represents the root mean square error of virtual target and real target drift between different frames in the motion state.

[0011] In one possible implementation, obtaining the evaluation index of the extended reality device based on the trajectory data of the virtual target and the real target includes: analyzing the dynamic virtual-real fusion image to obtain a dynamic angle difference index and a dynamic drift mean index, wherein the dynamic angle difference index characterizes the magnitude of the drift between the virtual target and the real target in different frames under the motion state, and the dynamic drift mean index characterizes the mean of the drift between the virtual target and the real target in different frames under the motion state; and obtaining the dynamic drift index based on the dynamic angle difference index and the dynamic drift mean index.

[0012] In one possible implementation, the dynamic virtual-real fusion image is a dynamic virtual-real fusion image captured frame by frame by the first camera when the extended reality device simulates the user's movement behavior while wearing the extended reality device.

[0013] In one possible implementation, the virtual-real fusion image is captured frame by frame by the first camera after the extended reality device performs an alignment operation, the alignment operation including aligning the virtual target in the virtual image with the real target in the real image in a partial direction.

[0014] In one possible implementation, after obtaining the evaluation index of the extended reality device, the method further includes: obtaining an evaluation result of the degree of integration between the virtual and real images of the extended reality device based on the evaluation index of the extended reality device; and outputting the evaluation result.

[0015] Secondly, embodiments of this application provide a testing apparatus for an extended reality device, comprising: a first acquisition module, configured to acquire frame-by-frame captured images of a virtual-real fusion display of the extended reality device, wherein the virtual-real fusion image is an image superimposed from a virtual image and a real image of the extended reality device, the virtual image including a virtual target, and the real image including an image captured by the extended reality device containing a real target; a second acquisition module, configured to acquire trajectory data of the virtual target and the real target based on the virtual-real fusion image; and an evaluation module, configured to obtain an evaluation index of the extended reality device based on the trajectory data of the virtual target and the real target, wherein the evaluation index is used to evaluate the degree of fusion between the virtual image and the real image of the extended reality device.

[0016] Thirdly, embodiments of this application provide a testing system, which includes an electronic device, a first camera, an extended reality device, a movable support, and a real target; the first camera and the extended reality device are both located on the movable support, with the first camera located at the eye point of the extended reality device, and the real target located on one side of the extended reality device; the extended reality device is used to capture images of the real target and display a virtual-real fusion image based on the images; the first camera is used to capture the virtual-real fusion image displayed by the extended reality device frame by frame; the electronic device is connected to the first camera and is used to perform various possible implementations of the first aspect described above.

[0017] In one possible implementation, the first camera and the augmented reality device are fixed to a movable support, wherein, in a static state, the first camera, the augmented reality device, and the movable support are all stationary; and in a moving state, the movable support drives the first camera and the augmented reality device to move.

[0018] In one possible implementation, before the first camera captures frame-by-frame the virtual-real fusion image displayed by the extended reality device, the extended reality device acquires a virtual image containing a virtual target and a real image containing a real target, and aligns the virtual target in the virtual image with the real target in the real image.

[0019] In one possible implementation, the extended reality device includes a second camera. The extended reality device is used to identify the spatial position of a real target through the second camera, obtain the pose information of the real target in the real scene relative to the extended reality device, and automatically adjust the position of the virtual target in the virtual scene based on the pose information so that the virtual target in the virtual scene is aligned with the real target in the real scene. The pose information includes the position parameters of the real target based on the coordinate system of the extended reality device.

[0020] Fourthly, embodiments of this application provide an electronic device, including: a memory and a processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, causing the processor to perform various possible implementations as described in the first aspect above.

[0021] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement various possible implementations of the first aspect above.

[0022] In a sixth aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements various possible implementations of the first aspect described above.

[0023] The extended reality device testing method, testing apparatus, testing system, electronic device, storage medium, and program product provided in this application acquire frame-by-frame images of the virtual-real fusion displayed by the extended reality device using a first camera. Based on these images, trajectory data of the virtual and real targets are obtained. Finally, evaluation indicators of the virtual-real fusion images of the extended reality device are derived from the trajectory data of the virtual and real targets. This achieves the effect of more accurately evaluating the degree of virtual-real fusion by recording the changes of the virtual and real targets frame by frame in real time, reducing evaluation errors caused by hardware precision limitations, and making the monitoring of the degree of virtual-real fusion more precise. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0025] Figure 1 A schematic diagram of a scenario for the testing method of the extended reality device provided in this application;

[0026] Figure 2 Flowchart of the testing method for the extended reality device provided in this application Figure 1 ;

[0027] Figure 3 Flowchart of the testing method for the extended reality device provided in this application Figure 2 ;

[0028] Figure 4 A schematic diagram of the testing method for extended reality devices under dynamic conditions provided in this application;

[0029] Figure 5 Flowchart of the testing method for the extended reality device provided in this application Figure 3 ;

[0030] Figure 6 A schematic diagram of the structure of the test apparatus for the extended reality device provided in this application;

[0031] Figure 7 A schematic diagram of the structure of the electronic device provided in this application.

[0032] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0034] In related technologies, the World Ultra HD Video Industry Alliance standard T / UWA 016—2022, "Technical Requirements and Test Methods for XR Virtual-Real Fusion Production Systems," specifies the technical requirements and test methods for XR virtual-real fusion production systems and their four subsystems. This method is mainly applied to the testing of virtual-real fusion composite videos. It uses a camera system to capture real-world scenes and employs a tracking system and a rendering system to synthesize virtual-real fusion XR videos. When synthesizing virtual-real fusion XR videos, a virtual scene needs to be built, and the physical camera positioning space needs to be calibrated with the virtual scene space. At this point, the real-world positioning system displays the corresponding image on an LED screen based on the camera position. The shooting system captures the performance area and the LED screen background to obtain real-world video. The rendering system receives the video output from the shooting system, overlays an AR scene, and then renders and outputs the synthesized virtual-real fusion video.

[0035] The methods used in these related technologies have the following problems: First, they are incompatible with the application scenarios. Although the methods describe in detail the testing methods for evaluating the degree of virtual-real fusion, they are from the developer's perspective, synthesizing virtual-real fused videos in a fixed scene using a positioning system, camera system, and preset steps, and then evaluating the virtual-real fusion process of the synthesized video. However, in actual XR device usage scenarios, in order to achieve the goal of matching the constructed virtual environment with the real scene in real time, it is necessary to perform real-time dynamic tracking and rendering based on the user's pose. Therefore, the current testing methods are not suitable for dynamic scenes.

[0036] Secondly, there is a lack of consideration for the optical system of XR devices. Related methods employ external camera systems to test the degree of virtual-real fusion while synthesizing video, providing the final effect range of the fused video from the hardware performance at the production end. However, they cannot provide information on the final display performance of the fused video. After the real-world image and the rendered virtual image of an XR device are merged, the degree of virtual-real fusion is limited by the performance of the optical system, such as lens distortion, field of view, and resolution. Current methods do not address these optical factors, making it impossible to comprehensively evaluate the virtual-real fusion of the image during actual use of XR devices; the testing chain is incomplete.

[0037] Finally, the user experience dimension is lacking. Evaluating the virtual-real fusion in XR devices like optical engine headsets requires considering the user's subjective experience and interactive feel. However, the relevant technical methods mainly focus on technical errors and latency, lacking a comprehensive assessment of the user experience.

[0038] Figure 1 A schematic diagram of a scenario for the testing method of the extended reality device provided in this application, such as... Figure 1 As shown, in the specific application scenario of this application, Figure 1 It is a test system viewed from above, and the test system includes electronic equipment ( Figure 1(Not shown in the image) Augmented Reality device 101, first camera 102, movable support 103 and reality target 104.

[0039] The electronic device is connected to the first camera 102, which is used to capture frame-by-frame the virtual-real fusion image displayed by the extended reality device. The real target is located on one side of the extended reality device. Optionally, the real target 104 is located directly in front of the extended reality device 101.

[0040] Extended Reality Device 101, also known as an XR device. Figure 1 The two arcs in the diagram represent part of the outer contour of the extended reality device 101. There may be one, two, or more extended reality devices 101; the following description uses one extended reality device as an example. Because different extended reality devices have different hardware links, in some embodiments, the testing method of this application can be used to compare the impact of these hardware link differences on the degree of virtual-real image fusion between different extended reality devices.

[0041] The first camera 102 can be a high-speed camera. The first camera is located at the eyepoint position of the extended reality device; in other words, the first camera 102 is fixed to the near-eye display eyebox position of the extended reality device 101. Figure 1 In this configuration, the eye can view the real target from bottom to top, therefore the near-eye display eyebox is located on the side of the extended reality device 101 furthest from the real target 104. The distance between the first camera 102 and the extended reality device 101 is the eye-point distance d. Optionally, the high-speed camera can achieve a maximum shooting frame rate of 2200fps, and the high-speed camera is paired with a short-focal-length camera lens with a focal length of 16mm. In some embodiments, the frame rate of the first camera 102 is higher than the display frame rate of the extended reality device.

[0042] The movable support 103 can be a one-dimensional rotary table. The augmented reality device 101 and the first camera 102 can be fixed to the movable support 103. In motion, the movable support 103 moves along... Figure 1 The direction indicated by the middle arrow (r) is reversed left and right, and the extended reality device 101 and the first camera 102 move synchronously with the movable support 103.

[0043] A marker is printed on the real-world target 104, which may be an AprilTag label. AprilTag is a visual benchmark system pioneered by the University of Michigan and its experimental team, and can be used for various tasks, including augmented reality, robotics, and camera calibration. In some embodiments, other types of labels may also be used; this application does not limit the type of marker.

[0044] In some embodiments, the movable support 103 is a horizontally placed rotating platform, on which the first camera 102 and the augmented reality device 101 are fixed. Initially, the first camera 102 is positioned 10cm from the center of the rotating platform's axis, with its optical axis pointing outwards along the radius of rotation. During testing, the platform is adjusted so that the first camera 102 is located within the eye-box area of ​​the augmented reality device 101.

[0045] In some embodiments, the actual target 104 is placed 50cm directly in front of the rotating platform. As those skilled in the art will know, 10cm and 50cm are merely choices made based on the testing environment and equipment, and other values ​​may be used, depending on actual needs, as long as the testing objective of this embodiment is achieved and accurate testing is facilitated. Specific details are not elaborated here.

[0046] This application proposes a testing method for an extended reality device, comprising: acquiring frame-by-frame images of a virtual-real fusion display of the extended reality device captured by a first camera, wherein the virtual-real fusion image is an image superimposed from a virtual image and a real image of the extended reality device, the virtual image including a virtual target, and the real image including an image captured by the extended reality device containing a real target; acquiring trajectory data of the virtual target and the real target based on the virtual-real fusion image; and obtaining an evaluation index for the extended reality device based on the trajectory data of the virtual target and the real target, wherein the evaluation index is used to evaluate the degree of fusion between the virtual image and the real image of the extended reality device.

[0047] By acquiring frame-by-frame images of the virtual-real fusion displayed on the extended reality device captured by the first camera, and obtaining trajectory data of the virtual and real targets based on the virtual-real fusion images, and finally obtaining evaluation indicators of the extended reality device based on the trajectory data of the virtual and real targets, this application can more accurately evaluate the degree of virtual-real fusion by recording the changes of the virtual and real targets frame by frame in real time, reducing evaluation errors caused by hardware precision limitations, and making the monitoring of the degree of virtual-real fusion more precise.

[0048] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0049] Figure 2 Flowchart of the testing method for the extended reality device provided in this application Figure 1 ,like Figure 2 As shown, the method includes:

[0050] S201. Obtain the virtual-real fusion image displayed by the extended reality device captured frame by frame by the first camera. The virtual-real fusion image is the image after superimposing the virtual image and the real image of the extended reality device. The virtual image includes a virtual target, and the real image includes an image containing a real target captured by the extended reality device.

[0051] In one possible implementation, the virtual-real fusion image includes: a static virtual-real fusion image displayed on a stationary extended reality device, captured frame by frame by the first camera; and / or a dynamic virtual-real fusion image displayed on a moving extended reality device, captured frame by frame by the first camera.

[0052] In this configuration, the first camera and the extended reality device are fixed to a movable support, and the first camera and the movable support are in a stationary state. In this stationary state, the first camera, the extended reality device, and the movable support are all stationary, and the first camera captures frame-by-frame static images of the augmented reality device displayed in this stationary state, creating a fusion of virtual and real visuals.

[0053] In motion mode, the movable support moves the first camera and the augmented reality device to simulate spatial movement of the head display. In motion mode, the first camera captures frame-by-frame dynamic virtual-real fusion images displayed by the moving augmented reality device.

[0054] In some embodiments, during a specific test, the rotating platform can be kept stationary while the first camera 102 captures a virtual target and a fusion of the virtual and real images of the target under video perspective for 2 minutes.

[0055] The rotating platform rotates back and forth within a range of ±60 degrees at low speed (10° / s), medium speed (50° / s), and high speed (90° / s). At the same time, the first camera 102 records the superimposed image of the virtual scene presented by the extended reality device and the real scene. Each rotation speed test lasts for 5 minutes.

[0056] It is understood that those skilled in the art would know that other shooting durations, rotation speeds, or test durations could be specified, and the specific choice could be made according to actual needs. This is just one example and will not be described in detail here.

[0057] S202. Based on the virtual-real fusion image, obtain the trajectory data of the virtual target and the real target;

[0058] In one possible implementation, OpenCV's open-source algorithms can be used to perform a series of operations on each captured frame, including grayscale conversion, edge detection, and contour detection, to determine the position and orientation of the marker. Based on the marker's position and orientation, the coordinates of the target points of the virtual and real targets at that moment can be obtained. These target points can be corner points or center points. Then, the time of each frame is used as the horizontal axis to plot a curve, obtaining the distribution of the virtual and real target trajectories. The trajectory data of the virtual and real targets can then be parsed from these distributions.

[0059] Before acquiring the trajectory data of the virtual and real targets, the OpenCV open-source algorithm can also calculate the four corner points of each candidate contour and verify whether it conforms to the characteristics of the target through geometric relationships. In this way, users can judge whether the target point coordinates of the identified target are correct based on the algorithm feedback, thus improving the accuracy of target identification.

[0060] S203. Based on the trajectory data of the virtual target and the real target, an evaluation index for the extended reality device is obtained. The evaluation index is used to evaluate the degree of fusion between the virtual and real images of the extended reality device.

[0061] In one possible implementation, the extended reality device acquires a real-world image containing a real target through a first function, which represents a real scene. Optionally, the first function is a video pass-through function. After acquiring the real-world image, the extended reality device can project it onto its display panel, allowing the user to perceive the real scene through the projected image.

[0062] The extended reality device renders and outputs a virtual image containing a virtual target through a second function. This second function can be Virtual Reality (VR). The virtual target is a real target mapped onto the virtual image. The rendered virtual image can be overlaid on the real image, thus forming a fusion of virtual and real visuals.

[0063] The first camera can be used to capture the virtual-real fusion image and transmit the virtual-real fusion image to an electronic device for analysis, thereby obtaining the trajectory data of the virtual target and the real target.

[0064] By analyzing the entire process of virtual-real image fusion, and with the first camera recording the changes of the virtual and real targets frame by frame in real time from the device's eye position, the embodiments of this application can more accurately analyze the trajectory data of the virtual and real targets, reduce the evaluation error caused by hardware precision limitations, and make the monitoring of the degree of virtual-real fusion more precise.

[0065] In some embodiments, the first function may also be an optical see-through (OST) function, which allows the acquisition of a real-world image. When acquiring a real-world image using the optical see-through function, the real-world scene can be directly seen through an optical component at the front of the display device; this optical component can be a transparent lens.

[0066] The following describes the testing method of the extended reality device in detail using the first function as a video see-through function as an example. However, those skilled in the art will understand that the testing method of the extended reality device can also be implemented when the first function is an optical see-through function. This application does not limit the specific implementation of the first function.

[0067] In one possible implementation, the virtual fusion image includes a static virtual-real fusion image, the trajectory data includes static trajectory data, and the evaluation metrics include static jitter rate and / or relative jitter error. Optionally, the static trajectory data includes the coordinate changes of the target point of the virtual target in a static state. This target point can be, for example, a center point, a corner point, or any point on the target. When the target point is a corner point, it can be, for example, the upper left corner point or the lower right corner point; when the target point is a center point, a pre-defined center point can be used directly, or the center point can be calculated from the corner point. It should be understood that the target point can be set according to actual needs, and this application does not limit the specific setting and selection of the target point.

[0068] When stationary, the first camera is turned on, and the frame rate of the first camera can be, for example, 90Hz. This frame rate is consistent with the frame rate of the display panel of the extended reality device, that is, the frame rate of the display panel of the extended reality device is also 90Hz.

[0069] In one possible implementation, the different frames in this application can be adjacent frames or skipped frames. Since the first camera captures frames one by one, multiple virtual and real images of different frames can be obtained. In the following embodiments, the calculation of evaluation indicators is explained using adjacent frames as an example. These adjacent frames are different frames, captured sequentially on a time scale.

[0070] In one possible implementation, the static jitter rate index represents the proportion of the jitter of the virtual target and the jitter of the real target between different frames in the static state that are greater than a preset first jitter threshold.

[0071] The stationary jitter rate index includes a first stationary jitter rate and a second stationary jitter rate, and the stationary jitter rate index can be expressed as follows:

[0072] η νr =N νr / N

[0073] η νst =N νst / N

[0074] Where, η νr η is the percentage of virtual target jitter between adjacent frames in a static state that exceeds a preset first jitter threshold, i.e., the first static jitter rate. νst The second static jitter rate is the percentage of real-world target jitter exceeding a preset first jitter threshold between adjacent frames in a static state. νr N is the number of frames in which the coordinates of a virtual target in a static virtual image change more than a preset first jitter threshold relative to the coordinates of the previous frame at the first camera's imaging pixels. νst The first jitter threshold is defined as the number of frames in which the coordinates of a real target in a static image change more than a preset first jitter threshold relative to the coordinates of the previous frame at the first camera's imaging pixels. Here, N represents the total number of frames captured by the first camera. Optionally, the first jitter threshold can be a size of one pixel.

[0075] In some examples, the first jitter rate of Extended Reality (AR) device A is 0.00348%, and the second jitter rate is 0; the first jitter rate of AR device B is 2.50%, and the second jitter rate is 0.55%. Therefore, the difference between the degree of change of the virtual target and the degree of change of the real target in AR device A is smaller than the difference between the degree of change of the virtual target and the degree of change of the real target in AR device B, indicating that the degree of virtual-real fusion of AR device A is better than that of AR device B.

[0076] In one possible implementation, the relative jitter error index characterizes the relative change of the virtual target and the relative change of the real target between different frames in the static state. The relative jitter error index can be expressed as follows:

[0077]

[0078] in, This represents the displacement vector of the center corner point of the virtual target in the virtual image of the extended reality device in adjacent frames on the pixel plane. Let N represent the displacement vector of the center corner point of the real target in the real image of the extended reality device in adjacent frames on the pixel plane, where N is the total number of frames captured by the first camera. Based on the displacement vectors of the center corner point of the virtual target in the virtual image of the extended reality device in adjacent frames on the pixel plane, the displacement vectors of the center corner point of the real target in the real image of the extended reality device in adjacent frames on the pixel plane, and N being the total number of frames captured by the first camera, the mean square error of the displacement vectors of each adjacent frame in the virtual and real images can be obtained as a static relative jitter error index.

[0079] In some examples, the relative jitter error index for device A is 0.0909, and the relative jitter error index for device B is 0.5249.

[0080] In one possible implementation, the virtual fusion image includes a dynamic virtual-real fusion image, the trajectory data includes dynamic trajectory data, and the evaluation metrics include a dynamic virtual-real stuttering rate metric and / or a dynamic drift metric. The dynamic virtual-real fusion image is a frame-by-frame dynamic virtual-real fusion image captured by the first camera when the extended reality device simulates the user's movement behavior while wearing the extended reality device.

[0081] The movable support is equipped with a turntable, the rotation period and rotation angle of which can be set as needed. In motion, the frame rate of the first camera remains consistent with the frame rate of the extended reality device's display panel. For example, in motion, the turntable reciprocates with a fixed period of 5.65 seconds and a fixed amplitude of 20°, and the first camera's recording time during this motion is 5 minutes.

[0082] In one possible implementation, the dynamic virtual-real stuttering rate index represents the proportion of virtual target jitter and real target jitter between different frames in the motion state that are less than a preset second jitter threshold.

[0083] The dynamic virtual-real stuttering rate index includes a first dynamic virtual-real stuttering rate and a second dynamic virtual-real stuttering rate, which can be expressed as follows:

[0084] ζ vr =M vr / N

[0085] ζ vst =M vst / N

[0086] Where, ζ vr The first dynamic virtual-real stutter rate is defined as the percentage of virtual target jitter between adjacent frames in the motion state that exceeds a preset second jitter threshold. vstM is the percentage of real-target jitter between adjacent frames in the motion state that exceeds a preset second jitter threshold, i.e., the second dynamic virtual-real stutter rate. vr M is the number of frames in which the coordinates of a virtual target in a moving virtual image, at the first camera's imaging pixels, change less than a preset second jitter threshold relative to the coordinates of the previous frame. vst The number of frames in a moving target image where the coordinates of the target in the current frame relative to the previous frame change less than a preset second jitter threshold at the first camera's imaging pixels is defined as N. These frames can also be called stuttering frames. N is the total number of frames captured by the first camera. Optionally, the second jitter threshold can be a size of 1 pixel.

[0087] In one possible implementation, the dynamic drift index characterizes the root mean square error of the drift between the virtual target and the real target between different frames in the motion state.

[0088] Figure 3 Flowchart of the testing method for the extended reality device provided in this application Figure 2 ,like Figure 3 As shown, the evaluation metrics for the extended reality device, derived from the trajectory data of the virtual and real targets, include:

[0089] S301. Analyze the dynamic virtual-real fusion image to obtain a dynamic angle difference index and a dynamic drift mean index. The dynamic angle difference index represents the magnitude of the drift between the virtual target and the real target in different frames under the motion state, and the dynamic drift mean index represents the mean of the drift between the virtual target and the real target in different frames under the motion state.

[0090] In one possible implementation, the dynamic virtual-real fusion image is analyzed to obtain a dynamic angle difference index. To do this, the pixel coordinate differences between the virtual and real images captured by the first camera can be calculated as follows:

[0091] Δpixel=loc vR -loc VST (loc = x, y, z)

[0092] Where, loc vR This represents the pixel coordinates (loc) in the x-direction of the virtual target in the virtual image captured by the first camera when the extended reality device displays a virtual image. VSTThis refers to the pixel coordinates of the real target in the x-direction in the real-world image captured by the first camera when the extended reality device displays the real-world image. Since the direction of motion during the test is x-direction, this pixel coordinate difference is relative to the x-direction. However, those skilled in the art will understand that the x-direction is the direction of one of the axes of the Cartesian coordinate system formed by x, y, and z. In practical applications, the applicable coordinate system can be selected as needed, and the pixel coordinate difference can also be relative to the y-direction or the z-direction. This application does not limit the specific direction of the pixel coordinates.

[0093] Figure 4 A schematic diagram illustrating the testing method for extended reality devices under dynamic conditions provided in this application, as shown below. Figure 4 As shown, the real target and the virtual target are aligned along the x-direction, with the real target positioned above the virtual target. After obtaining the pixel coordinate differences between the virtual and real images captured by the first camera, these pixel coordinate differences can be converted into angular differences based on the intrinsic parameters of the first camera. The specific process is as follows:

[0094]

[0095] Among them, c x Let f be the coordinates of the optical center of the target surface of the first camera. x θ represents the focal length of the first camera in the pinhole model; both are intrinsic parameters of the first camera. vR To extend the angle θ of the virtual target in the x-direction in the virtual image captured by the first camera when the real device displays a virtual image, vsT To extend the angle of the real target in the x-direction in the real image captured by the first camera when the reality device displays the real image, Δθ is the angle difference, i.e., the dynamic angle difference index.

[0096] Since the direction of motion in the test is the x-direction, the angle difference is relative to the x-direction. However, those skilled in the art will understand that the x-direction is the direction of one of the axes of the Cartesian coordinate system formed by x, y, and z. In practical applications, the applicable coordinate system can be selected as needed, and the angle difference can also be relative to the y-direction or the z-direction. This application does not limit the specific direction of the angle. When the angle difference is relative to the y-direction, x in the above formula can be replaced with y; when the angle difference is relative to the z-direction, x in the above formula can be replaced with z.

[0097] S302. Based on the dynamic angle difference index and the dynamic drift mean index, the dynamic drift index is obtained.

[0098] In one possible implementation, the dynamically fused virtual-real image is analyzed to obtain a dynamic drift mean index. The dynamic drift mean index can be expressed as follows:

[0099]

[0100] Where, Δθ - Let i represent the angular difference in the i-th frame. Since the first camera captures images frame by frame, there can be multiple dynamic angular difference indices. The average of these multiple dynamic angular difference indices yields the dynamic drift mean index Δθ-mean.

[0101] In one possible implementation, the dynamic drift index is obtained based on the dynamic angle difference index and the dynamic drift mean index, and can be expressed as follows:

[0102]

[0103] Where Δθ-i is the i-th dynamic angle difference index, Δθ _ RMSE is a dynamic drift metric. In some embodiments, the evaluation metric may further include a dynamic drift range metric, which represents the range of drift between virtual and real targets in adjacent frames during the motion state, and can be expressed as follows:

[0104] Δθ-range=Δθ-max-Δθ-min

[0105] Wherein, Δθ-max is the maximum dynamic angle difference index under motion, and Δθ-min is the minimum dynamic angle difference index under motion.

[0106] In some embodiments, different frames in this application can also be skipped frames. A skipped frame refers to two frames that are not adjacent in time, such as the first frame and the third frame. Since the above formula is calculated using adjacent frames as an example, if it is necessary to calculate the evaluation index of non-adjacent skipped frames, the indices of different adjacent frames can be summed to obtain the index of non-adjacent skipped frames. It is understood that this application does not limit whether different frames are adjacent.

[0107] In some examples, both the virtual and real-world images have a pixel size of 1920x1088, and the intrinsic parameter matrix K of the first camera is:

[0108] From the intrinsic parameter matrix K, we know that the focal length component in the x-direction is fx = 1771.4, the focal length component in the y-direction is fy = 1771.18, the principal point (optical center) has coordinates cx = 958.93 in the x-direction, and coordinates cy = 530.86 in the y-direction. Therefore, the spatial angle corresponding to 1 pixel on the imaging plane of the first camera is calculated as follows:

[0109] Figure 5 Flowchart of the testing method for the extended reality device provided in this application Figure 3,like Figure 5 As shown, after obtaining the evaluation metrics for the extended reality device, the method further includes:

[0110] S501. Based on the evaluation indicators of the extended reality device, obtain the evaluation result of the degree of integration between the virtual and real images of the extended reality device;

[0111] In one possible implementation, an evaluation index under static conditions can be used to generate an evaluation result of the degree of fusion between the virtual and real images of the extended reality device. Alternatively, an evaluation index under motion conditions can be used to generate an evaluation result of the degree of fusion between the virtual and real images of the extended reality device. Or, a combination of evaluation indexes under static and motion conditions can be used to generate an evaluation result of the degree of fusion between the virtual and real images of the extended reality device.

[0112] S502. Output the evaluation results.

[0113] In one possible implementation, the evaluation result can be a number or a graded result such as "Excellent," "Good," or "Poor." The evaluation result can be displayed on the screen of an electronic device. Because it achieves a multi-dimensional analysis of the virtual-real fusion problem, including in-depth exploration from multiple aspects such as hardware, mechanical motion, and image jitter, it can identify potential technical problems in the system chain, such as system errors or algorithm mismatches, and provide a clear direction for problem-solving.

[0114] Furthermore, by taking into account the impact of the extended reality device's optical system on the degree of virtual-real fusion and incorporating comprehensive indicators related to user experience, the embodiments of this application can improve the accuracy of evaluating the degree of virtual-real fusion of extended reality devices.

[0115] It should be noted that the above embodiments are illustrated using the first function as a video see-through function as an example. When the first function is an optical see-through function, since the optical components are transparent, the real scene can be directly seen. Therefore, when it comes to indicators related to the real scene, the relevant parameters of the real scene can be obtained directly without further calculation. Only the relevant parameters related to the virtual scene need to be calculated, which greatly reduces the workload of calculating evaluation indicators and improves the calculation efficiency of evaluation indicators.

[0116] In one possible implementation, the fused virtual-real image is captured frame-by-frame by the first camera after the extended reality device performs an alignment operation. The alignment operation includes aligning the virtual target in the virtual image with the real target in the real image in certain directions. This alignment allows for comprehensive analysis of the alignment of the virtual and real targets under different states.

[0117] The real-world target can be located at the center of the real-world image. The marker on the real-world target can be square in shape, and when the real-world target is in the center of the real-world image, the marker is also in the center of the real-world image. The marker can be composed of black and white square patterns, and each marker has a unique ID. The design of the markers ensures that they can be effectively identified under different lighting conditions.

[0118] In some possible implementations, the extended reality device can be controlled using Unity-based software or algorithms. When the extended reality device is activated, it can capture a realistic image reflecting the actual scene using the first function, and the real target in the real image can then be seen on the extended reality device's display panel. Simultaneously, the extended reality device can generate a virtual target based on Unity's software algorithms, whose shape matches the real target. However, since there is a positional discrepancy between the virtual and real targets, it is necessary to align the virtual and real targets in certain directions.

[0119] In one possible implementation, the virtual target and the real target can be aligned in some directions using automated alignment, or they can be aligned in some directions using manual alignment.

[0120] When using automated alignment, the augmented reality device can identify the spatial position of the real target using its built-in second camera, obtaining the pose information of the real target relative to the augmented reality device in the real-world image. The pose information includes the position parameters of the real target based on the coordinate system of the augmented reality device, and these position parameters can be coordinates, translation, rotation, etc. The second camera can be a binocular camera.

[0121] After obtaining the pose information of the real target, since the virtual target is generated by software, the position of the virtual target in the virtual image can be automatically adjusted by adjusting the software's own parameters, thereby aligning the virtual target in the virtual image with the real target in the real image in certain directions. Specific parameters are selected according to actual needs and will not be elaborated here.

[0122] In one possible implementation, the virtual target and the real target can be aligned in some directions by adjusting the center point of the virtual target, or by adjusting the corner points of the virtual target. It is understood that this application does not limit whether the reference point used in the alignment process is the center point or a corner point.

[0123] By using the second camera built into the augmented reality device and combining it with the SLAM positioning system, the embodiments of this application can more accurately capture the relative position information between the augmented reality device and the target, thereby improving the accuracy and reliability of positioning.

[0124] After aligning the virtual target in the virtual image with the real target in the real image in a partial direction, the extended reality device can acquire a real image containing the real target under the first function, and output a virtual image containing the virtual target through rendering under the second function.

[0125] In one possible implementation, the virtual target in the virtual image can be manually aligned with the real target in the real image in certain directions. The specific process of manual alignment is as follows: by adjusting the position parameters of the virtual target and continuously moving it, the position is determined by subjective judgment of the human eye or by objective multi-angle shooting analysis until the virtual target in the virtual image is aligned with the real target in the real image in certain directions.

[0126] After the extended reality device displays the real scene and the virtual scene simultaneously, the first camera begins to capture the virtual-real fusion image including the real scene and the virtual scene, and then transmits the virtual fusion image to the electronic device for analysis.

[0127] Figure 6 A schematic diagram of the structure of the test apparatus for the extended reality device provided in this application is shown below. Figure 6 As shown, the extended reality device testing apparatus 60 provided in this embodiment includes:

[0128] The first acquisition module 601 is used to acquire the virtual-real fusion image displayed by the extended reality device captured frame by frame by the first camera. The virtual-real fusion image is the image after superimposing the virtual image and the real image of the extended reality device. The virtual image includes a virtual target, and the real image includes an image containing a real target captured by the extended reality device.

[0129] The second acquisition module 602 is used to acquire trajectory data of the virtual target and the real target based on the virtual-real fusion image;

[0130] The evaluation module 603 is used to obtain the evaluation index of the extended reality device based on the trajectory data of the virtual target and the real target. The evaluation index is used to evaluate the degree of fusion between the virtual and real images of the extended reality device.

[0131] In one possible implementation, the virtual-real fusion image includes: a static virtual-real fusion image displayed on a stationary extended reality device, captured frame by frame by the first camera; and / or a dynamic virtual-real fusion image displayed on a moving extended reality device, captured frame by frame by the first camera.

[0132] In one possible implementation, the virtual fusion image includes a static virtual-real fusion image, the trajectory data is static trajectory data, and the evaluation index includes static jitter rate and / or relative jitter error; the static jitter rate index represents the proportion of virtual target jitter and real target jitter greater than a preset first jitter threshold between different frames in the static state; the relative jitter error index represents the relative change of virtual target and real target between different frames in the static state.

[0133] In one possible implementation, the virtual fusion image includes a dynamic virtual-real fusion image, the trajectory data is dynamic trajectory data, and the evaluation index includes a dynamic virtual-real stuttering rate index and / or a dynamic drift index; the dynamic virtual-real stuttering rate index represents the proportion of virtual target jitter and real target jitter less than a preset second jitter threshold between different frames in the motion state; the dynamic drift index represents the root mean square error of virtual target and real target drift between different frames in the motion state.

[0134] In one possible implementation, the evaluation module includes: a dynamic analysis module, used to analyze the dynamic virtual-real fusion image to obtain a dynamic angle difference index and a dynamic drift mean index, wherein the dynamic angle difference index characterizes the magnitude of the drift between the virtual target and the real target in different frames under the motion state, and the dynamic drift mean index characterizes the mean of the drift between the virtual target and the real target in different frames under the motion state; and an error calculation module, used to obtain the dynamic drift index based on the dynamic angle difference index and the dynamic drift mean index.

[0135] In one possible implementation, the dynamic virtual-real fusion image is a dynamic virtual-real fusion image captured frame by frame by the first camera when the extended reality device simulates the user's movement behavior while wearing the extended reality device.

[0136] In one possible implementation, the virtual-real fusion image is captured frame by frame by the first camera after the extended reality device performs an alignment operation, the alignment operation including aligning the virtual target in the virtual image with the real target in the real image in a partial direction.

[0137] In one possible implementation, after obtaining the evaluation index of the extended reality device, the testing device for the extended reality device further includes: an evaluation result acquisition module, used to acquire an evaluation result of the degree of fusion between the virtual and real images of the extended reality device based on the evaluation index of the extended reality device; and an output module, used to output the evaluation result.

[0138] The electronic device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0139] Furthermore, this application also provides a testing system, which includes an electronic device, a first camera, an extended reality device, a movable support, and a real target; the first camera and the extended reality device are both located on the movable support, with the first camera located at the eye point of the extended reality device, and the real target located on one side of the extended reality device; the extended reality device is used to capture images of the real target and display a virtual-real fusion image based on the images; the first camera is used to capture the virtual-real fusion image displayed by the extended reality device frame by frame; the electronic device is connected to the first camera and is used to perform various possible implementations of the first aspect above.

[0140] In one possible implementation, the first camera and the augmented reality device are fixed to a movable support, wherein, in a static state, the first camera, the augmented reality device, and the movable support are all stationary; and in a moving state, the movable support drives the first camera and the augmented reality device to move.

[0141] In one possible implementation, before the first camera captures frame-by-frame the virtual-real fusion image displayed by the extended reality device, the extended reality device acquires a virtual image containing a virtual target and a real image containing a real target, and aligns the virtual target in the virtual image with the real target in the real image.

[0142] In one possible implementation, the extended reality device includes a second camera. The extended reality device is used to identify the spatial position of a real target through the second camera, obtain the pose information of the real target in the real scene relative to the extended reality device, and automatically adjust the position of the virtual target in the virtual scene based on the pose information so that the virtual target in the virtual scene is aligned with the real target in the real scene. The pose information includes the position parameters of the real target based on the coordinate system of the extended reality device.

[0143] Figure 7 A schematic diagram of the structure of the electronic device provided in this application. Figure 7As shown, the electronic device 70 provided in this embodiment includes at least one processor 701 and a memory 702. Optionally, the device 70 further includes a communication component 703. The processor 701, memory 702, and communication component 703 are connected via a bus 704.

[0144] In a specific implementation, at least one processor 701 executes computer execution instructions stored in memory 702, causing at least one processor 701 to perform the above-described method.

[0145] The specific implementation process of processor 701 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0146] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0147] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0148] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0149] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0150] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0151] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0152] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0153] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0154] The units described as dividing components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0156] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0158] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A testing method for an extended reality device, characterized in that, The first camera is located at the eye point of the augmented reality device; the method includes: The virtual-real fusion image displayed on the extended reality device is captured frame by frame by the first camera. The virtual-real fusion image is the superimposed image of the virtual image and the real image of the extended reality device. The virtual image includes a virtual target, and the real image includes an image containing a real target captured by the extended reality device. Based on the virtual-real fusion image, the trajectory data of the virtual target and the real target are obtained; Based on the trajectory data of the virtual target and the real target, an evaluation index for the extended reality device is obtained. The evaluation index is used to evaluate the degree of fusion between the virtual and real images of the extended reality device.

2. The method according to claim 1, characterized in that, The virtual-real fusion image includes: The first camera captures frame-by-frame static images of the augmented reality device displayed while it is stationary; and / or, The first camera captures frame-by-frame dynamic virtual-real fusion images displayed on a moving extended reality device.

3. The method according to claim 2, characterized in that, The virtual fusion image includes a static virtual-real fusion image, the trajectory data is static trajectory data, and the evaluation index includes static jitter rate and / or relative jitter error; The static jitter rate index represents the proportion of virtual target jitter and real target jitter exceeding a preset first jitter threshold between different frames in the static state. The relative jitter error index characterizes the relative changes of the virtual target and the real target between different frames in the static state.

4. The method according to claim 2, characterized in that, The virtual fusion screen includes a dynamic virtual-real fusion screen, the trajectory data is dynamic trajectory data, and the evaluation indicators include dynamic virtual-real stuttering rate and / or dynamic drift indicator. The dynamic virtual-real stuttering rate index represents the percentage of virtual target jitter and real target jitter between different frames under the motion state that are less than a preset second jitter threshold. The dynamic drift index characterizes the root mean square error of the drift between the virtual target and the real target in different frames under the motion state.

5. The method according to claim 4, characterized in that, The evaluation metrics for the extended reality device, derived from the trajectory data of the virtual and real targets, include: The dynamic virtual-real fusion image is analyzed to obtain a dynamic angle difference index and a dynamic drift mean index. The dynamic angle difference index represents the magnitude of the drift between the virtual target and the real target in different frames under the motion state, and the dynamic drift mean index represents the mean of the drift between the virtual target and the real target in different frames under the motion state. The dynamic drift index is obtained based on the dynamic angle difference index and the dynamic drift mean index.

6. The method according to claim 2, characterized in that, The dynamic virtual-real fusion image is a dynamic virtual-real fusion image captured frame by frame by the first camera when the extended reality device simulates the user's movement behavior while wearing the extended reality device.

7. The method according to any one of claims 1-5, characterized in that, The virtual-real fusion image is captured frame by frame by the first camera after the extended reality device performs an alignment operation. The alignment operation includes aligning the virtual target in the virtual image with the real target in the real image in a certain direction.

8. The method according to any one of claims 1-6, characterized in that, After obtaining the evaluation metrics for the extended reality device, the method further includes: Based on the evaluation metrics of the extended reality device, obtain the evaluation result of the degree of integration between the virtual and real images of the extended reality device; Output the evaluation results.

9. A testing apparatus for an extended reality device, characterized in that, include: The first acquisition module is used to acquire the virtual-real fusion image displayed by the extended reality device captured frame by frame by the first camera. The virtual-real fusion image is the image after superimposing the virtual image and the real image of the extended reality device. The virtual image includes a virtual target, and the real image includes an image containing a real target captured by the extended reality device. The second acquisition module is used to acquire trajectory data of the virtual target and the real target based on the virtual-real fusion image; The evaluation module is used to obtain evaluation indicators for the extended reality device based on the trajectory data of the virtual target and the real target. The evaluation indicators are used to evaluate the degree of fusion between the virtual and real images of the extended reality device.

10. A testing system, characterized in that, The testing system includes electronic devices, a first camera, an augmented reality device, a movable support, and a real target. Both the first camera and the augmented reality device are located on the movable support, with the first camera located at the eye point of the augmented reality device and the real target located on one side of the augmented reality device. The extended reality device is used to capture images of the real target and display a virtual-real fusion image based on the images; The first camera is used to capture frame-by-frame images of the virtual-real fusion displayed on the extended reality device; The electronic device is connected to the first camera and is used to perform the method as described in any one of claims 1-8.

11. The testing system according to claim 10, characterized in that, The first camera and the augmented reality device are fixed to a movable bracket. In a static state, the first camera, the augmented reality device, and the movable bracket are all stationary. In a moving state, the movable bracket drives the first camera and the augmented reality device to move.

12. The testing system according to claim 10, characterized in that, Before the first camera captures frame-by-frame the virtual-real fusion image displayed by the extended reality device, the extended reality device acquires a virtual image containing a virtual target and a real image containing a real target, and aligns the virtual target in the virtual image with the real target in the real image.

13. The testing system according to claim 11, characterized in that, The extended reality device includes a second camera. The extended reality device is used to identify the spatial position of the real target through the second camera, obtain the pose information of the real target in the real image relative to the extended reality device, and automatically adjust the position of the virtual target in the virtual image based on the pose information so that the virtual target in the virtual image is aligned with the real target in the real image. The pose information includes the position parameters of the real target based on the coordinate system of the extended reality device.

14. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

16. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-8.

Images