Evaluation method and device for pose track of VR device

Abstract

The invention relates to the technical field of VR (Virtual Reality), and provides a method and equipment for evaluating a pose track of VR equipment. According to the method, the pose track of the target VR equipment is comprehensively evaluated by building the Vcon motion capture system and the mechanical arm system which can be operated by a single person, the method can be suitable for evaluation of the positioning and attitude determination algorithm precision of the VR equipment in different motion states, the application range is wider, the equipment cost is low, and operation is easy. Wherein aiming at the problem that a pose track observed by a system in each motion state is inconsistent with a pose track timestamp, a coordinate system and a scale of equipment, timestamp alignment is carried out according to respective angular velocity modulus length sequences of two pose tracks calculated at a preset time interval; according to the relative relation between the two coordinate systems at different moments established according to the two pose tracks, space alignment and scale alignment are carried out, the positioning precision of the VR equipment can be accurately evaluated after alignment, and reference is provided for design and safe use of related products.

Description

Technical Field

[0001] The present application relates to the field of virtual reality (VR) technology, and provides a method and device for evaluating the posture trajectory of a VR device. Background Art

[0002] As a new visual display technology, VR technology allows users to immerse themselves in a virtual three-dimensional environment for interaction through a head-mounted display (HMD) and hand controllers, bringing people a new visual experience and satisfying people's pursuit of new ways of presenting information.

[0003] In recent years, with the introduction of the concept of the metaverse and the improvement of mobile computing capabilities, users' requirements for free interaction in the field of VR technology have gradually increased. Therefore, key performance parameters such as the 6-degree-of-freedom (DoF) positioning and attitude determination technology, tracking accuracy, and stability of the user's head on the mobile platform have become direct evaluation indicators that directly affect the user's actual interactive experience. At the same time, the rapid and accurate evaluation of the accuracy of the posture trajectory of mobile VR devices also provides an important reference for the design of VR products and the development of related software and hardware. However, there is currently a lack of an accurate evaluation method for the accuracy of the posture trajectory of different VR devices that is simple to operate, cost-controlled, and applicable. Summary of the invention

[0004] The embodiments of the present application provide a method and device for evaluating the posture trajectory of a VR device for realizing convenient, low-cost, and wide-range posture trajectory evaluation.

[0005] On the one hand, an embodiment of the present application provides a method for evaluating a posture trajectory of a VR device, comprising a Vicon motion capture system and a robotic arm system, wherein the Vicon motion capture system comprises a plurality of fixed infrared cameras, a target VR device with a plurality of light-sensitive points attached thereto performs a first movement within the field of view of the plurality of infrared cameras, the robotic arm system comprises a checkerboard calibration plate and a movable robotic arm, the target device is fixed to the end of the robotic arm and performs a second movement with the robotic arm, and the method comprises:

[0006] Obtaining a first pose trajectory of a rigid body centroid formed by a plurality of light-sensitive points acquired by the Vicon motion capture system in a first motion state and a second pose trajectory of the target VR device itself, and obtaining a third pose trajectory of the target VR device acquired by the manipulator system in a second motion state and a fourth motion trajectory of the target VR device itself;

[0007] For the two posture trajectories under each motion state, the angular velocity modulus is calculated at preset time intervals to obtain two angular velocity modulus sequences, and the timestamps of the two posture trajectories are aligned according to the two angular velocity modulus sequences;

[0008] For the two posture trajectories with aligned timestamps in each motion state, a relative relationship between the two coordinate systems at different times is established, and the two posture trajectories are spatially aligned and scale-aligned according to the relative relationship;

[0009] For each motion state, the difference between the two pose trajectories after the timestamps, space and scale are aligned is calculated;

[0010] The positioning accuracy of the target VR device is comprehensively evaluated based on each group of differences.

[0011] On the other hand, an embodiment of the present application provides an evaluation device, including a processor, a memory, and a communication interface, wherein the communication interface, the memory, and the processor are connected via a bus;

[0012] The memory stores a computer program, and the processor performs the following operations according to the computer program:

[0013] Through the communication interface, a first pose trajectory of the rigid body center of mass formed by multiple photosensing points collected by the Vicon motion capture system in the first motion state and a second pose trajectory of the target VR device itself are obtained, and a third pose trajectory of the target VR device collected by the robotic arm system in the second motion state and a fourth motion trajectory of the target VR device itself are obtained; wherein, the Vicon motion capture system includes multiple fixed infrared cameras, and the target VR device with multiple photosensing points attached thereto performs a first motion within the field of view of the multiple infrared cameras, and the robotic arm system includes a checkerboard calibration plate and a movable robotic arm, and the target device is fixed to the end of the robotic arm and performs a second motion with the robotic arm;

[0014] For the two posture trajectories under each motion state, the angular velocity modulus is calculated at preset time intervals to obtain two angular velocity modulus sequences, and the timestamps of the two posture trajectories are aligned according to the two angular velocity modulus sequences;

[0015] For the two posture trajectories with aligned timestamps in each motion state, a relative relationship between the two coordinate systems at different times is established, and the two posture trajectories are spatially aligned and scale-aligned according to the relative relationship;

[0016] For each motion state, the difference between the two pose trajectories after the timestamps, space and scale are aligned is calculated;

[0017] The positioning accuracy of the target VR device is comprehensively evaluated based on each group of differences.

[0018] Optionally, the processor aligns the timestamps of the two pose trajectories according to the two angular velocity modulus length sequences, and the specific operation is:

[0019] Sliding a first angular velocity modulus length sequence on a second angular velocity modulus length sequence, wherein the length of the first angular velocity modulus length sequence is shorter than the length of the second angular velocity modulus length sequence;

[0020] After each sliding, calculating the Pearson correlation coefficient between the first angular velocity modulus length sequence and the second angular velocity modulus length sequence;

[0021] When the Pearson correlation coefficient is the largest, determining the starting time difference between two posture trajectories according to the first angular velocity modulus length sequence and the second angular velocity modulus length sequence;

[0022] According to the starting time difference, the timestamps of the two pose trajectories are aligned.

[0023] Optionally, the second posture trajectory of the two posture trajectories in the first motion state and the fourth posture trajectory of the two posture trajectories in the second motion state are recorded as trajectories to be evaluated, the first posture trajectory of the two posture trajectories in the first motion state and the third posture trajectory in the second motion state are recorded as true value trajectories, and the processor calculates the angular velocity modulus in the following manner:

[0024] According to the trajectory to be evaluated, obtaining the estimated posture recorded by the target VR device itself at the first moment and the second moment, and, according to the true value trajectory, obtaining the real posture observed by the system at the first moment and the second moment;

[0025] Determine a first transformation matrix including rotation information and translation information according to the estimated postures at the first moment and the second moment, and determine a second transformation matrix including rotation information and translation information according to the actual postures at the first moment and the second moment;

[0026] The rotation information and the translation information in the first transformation matrix are represented separately, and a first angular velocity vector is obtained by deriving the rotation information, and the rotation information and the translation information in the second transformation matrix are represented separately, and a second angular velocity vector is obtained by deriving the rotation information;

[0027] A first angular velocity modulus is obtained according to the first angle vector, and a second angular velocity modulus is obtained according to the second angular velocity vector.

[0028] Optionally, the coordinate systems in the Vicon motion capture system and the robotic arm system have the following corresponding relationship:

[0029] The first reference coordinate system in which the target VR device in the Vicon motion capture system records its own second posture trajectory corresponds to the second reference coordinate system in which the target VR device in the robotic arm system records its own fourth posture trajectory;

[0030] The first world coordinate system of the first pose trajectory of the rigid body center of mass formed by the multiple photosensing points acquired by the Vicon motion capture system corresponds to the second world coordinate system of the third pose trajectory of the end of the robotic arm acquired by the robotic arm system;

[0031] The local coordinate system of the target VR device in the Vicon motion capture system corresponds to the local coordinate system of the target VR device in the robotic arm system;

[0032] The rigid body centroid coordinate system formed by multiple photosensing points on the target VR device in the Vicon motion capture system corresponds to the robotic arm end coordinate system in the robotic arm system.

[0033] Optionally, the second posture trajectory of the two posture trajectories in the first motion state and the fourth posture trajectory of the two posture trajectories in the second motion state are recorded as trajectories to be evaluated, and the first posture trajectory of the two posture trajectories in the first motion state and the third posture trajectory in the second motion state are recorded as true value trajectories;

[0034] The processor establishes a relative relationship between two coordinate systems at different times for two posture trajectories with time stamps aligned in each motion state, and performs spatial alignment and scale alignment on the two posture trajectories according to the relative relationship. The specific operations are as follows:

[0035] For the estimated trajectory and the true trajectory with aligned timestamps under each motion state, a relative relationship between the reference coordinate system of the estimated trajectory and the world coordinate system of the true trajectory is established according to the 3D points on the estimated trajectory and the projection points of the 3D points on the true trajectory;

[0036] According to the invariance principle of the relative relationship between two coordinate systems at different times, the transformation matrix equation is established;

[0037] According to two points on the trajectory to be evaluated and the true value trajectory at the same time, solving the transformation matrix between the coordinate system where the trajectory to be evaluated is located and the coordinate system where the true value trajectory is located, and the scale scaling coefficient between the trajectory to be evaluated and the true value trajectory;

[0038] The trajectory to be evaluated and the true trajectory are spatially aligned according to the transformation matrix, and the trajectory to be evaluated and the true trajectory are scale-aligned according to the scale scaling coefficient.

[0039] Optionally, the Vicon motion capture system, the robotic arm system, and the target VR device each correspond to an interface function, and the interface function is used to obtain and store corresponding posture trajectories, and the storage path of each posture trajectory is written into the same configuration file, so that the Vicon motion capture system and the robotic arm system use the same set of software programs to evaluate the positioning accuracy of the target VR device according to different parameters in the configuration file.

[0040] Optionally, the evaluation index of the difference between the two posture trajectories includes at least one of the root mean square error, error mean, mean error, standard deviation, minimum error and maximum error of the absolute trajectory error.

[0041] Optionally, the target VR device is any one of a head-mounted display HMD and a handle.

[0042] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer device to execute the steps of a method for evaluating a VR device posture trajectory provided in an embodiment of the present application.

[0043] The beneficial effects of the method for evaluating the posture trajectory of a VR device provided in the embodiment of the present application are as follows:

[0044] The Vicon motion capture system and robotic arm system built with multiple infrared cameras are used to evaluate the pose trajectory of the target VR device. The operation process is simple and the equipment cost is low. During the evaluation process, the angular velocity modulus of the two pose trajectories is calculated at preset time intervals to solve the problem of inconsistency between the pose trajectory observed by the system and the pose trajectory of the device itself in each motion state, and two angular velocity modulus sequences are obtained. The timestamps of the two pose trajectories are aligned according to the two angular velocity modulus sequences. In addition, the pose trajectory observed by the system in each motion state is inconsistent with the pose trajectory of the device itself in space and scale. The relative relationship between the two coordinate systems at different times established by the two pose trajectories is used to perform spatial alignment and scale alignment. In this way, the difference between the two aligned pose trajectories can be calculated, so as to achieve a comprehensive evaluation of the positioning accuracy of the target VR device by the Vicon motion capture system and the robotic arm system. Since the device has a small motion range and a single posture in the robotic arm system, and a large motion range and flexible posture in the Vicon motion capture system, the two systems are used to comprehensively evaluate the positioning accuracy of the VR device, which can meet the needs of VR devices in different scenarios and has a wider range of applications.

[0045] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 A schematic diagram of a VR device provided in an embodiment of the present application;

[0048] Figure 2 An evaluation system for the posture trajectory of a VR device provided in an embodiment of the present application;

[0049] Figure 3A A schematic diagram of an HMD with a photosensor attached thereto provided in an embodiment of the present application;

[0050] Figure 3B A schematic diagram of a handle with a photosensitive point attached thereto provided in an embodiment of the present application;

[0051] Figure 4This is an architecture diagram of a VR device posture trajectory evaluation method provided in an embodiment of the present application;

[0052] Figure 5 A flow chart of a method for evaluating a posture trajectory of a VR device provided in an embodiment of the present application;

[0053] Fig. 6A A schematic diagram of the coordinate system of the Vicon motion capture system provided in an embodiment of the present application;

[0054] Figure 6B A schematic diagram of a coordinate system of a robotic arm system provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of the principle of aligning the timestamps of the trajectory to be evaluated and the true value trajectory provided in an embodiment of the present application;

[0056] Figure 8 A flow chart of a method for aligning timestamps of two pose trajectories provided in an embodiment of the present application;

[0057] Fig. 9 A flow chart of a method for calculating angular velocity modulus provided in an embodiment of the present application;

[0058] Fig.10 A flow chart of a method for aligning the space and scale of two pose trajectories provided in an embodiment of the present application;

[0059] Fig.11 A structural diagram of the evaluation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0061] Generally, VR devices include HMD and two-handed controllers, such as Figure 1 As shown, the HMD is used to display the virtual space, and the user interacts with the virtual world by operating the handle.

[0062] Considering that most of the usage scenarios of VR devices are in indoor scenes with limited space, the evaluation task of 6DoF positioning and posture trajectory accuracy of mobile VR devices requires continuous, stable and accurate tracking of the movement of VR devices, and rapid output of quantifiable evaluation results.

[0063] At present, some mainstream evaluation methods on the market include evaluation methods based on ultra-precision optical tracking systems such as ART (Advanced Realtime Tracking) and evaluation methods using Outside-in spatial positioning monitoring technology. Among them, although the evaluation methods based on ultra-precision optical tracking systems such as ART have high accuracy, they are not suitable for small scenes with limited space, and the operation is complicated and the installation cost is high, which is unbearable for some small teams; while in the evaluation method using Outside-in positioning technology, it is necessary to use external sources (such as infrared laser transmitters) to locate and determine the attitude of VR devices, and corresponding sensors (such as infrared photosensitive devices) must also be installed on VR devices, which is costly and has poor scalability. With the development of Simultaneous Localization and Mapping (SLAM) technology, more and more VR devices use SLAM technology that integrates multi-source sensor data to locate themselves and perceive the surrounding environment. In the absence of external precise trajectories, according to the loop detection algorithm of SLAM, the accuracy of the positioning algorithm can be judged according to the degree of closure of the trajectory when there is a loop in the trajectory. Although this method is widely applicable to scenarios, it has low accuracy.

[0064] In order to accurately track and evaluate the accuracy of spatial parameters such as the position and posture of a 6DoF VR device in an environment with a small activity space (such as indoors), an embodiment of the present application provides a method for evaluating the posture trajectory of a VR device. The method builds a Vicon motion capture system and a robotic arm system that can be operated by one person to obtain an accurate posture trajectory of the VR device, and compares it with the posture trajectory output by the VR device itself, so as to comprehensively evaluate the performance of the positioning and posture determination algorithm in the VR device, and provide a reference for the design and safe use of related products. In addition, since the VR device in the robotic arm system has a small motion range and a single posture, while the VR device in the Vicon motion capture system has a large motion range and a flexible posture, the two systems are used to evaluate the VR device separately, which can be applicable to the use requirements of the VR device in different scenarios and has a wider range of applications. Among them, in the evaluation process, in order to solve the problem of inconsistent timestamps of posture trajectories obtained by different sensors, this method obtains the angular velocity modulus at the same frequency for the two posture trajectories in each motion state based on Lie algebra theory, and calculates the Pearson correlation coefficient to obtain the starting time difference of the two posture sequences, so as to align the two posture trajectories in time. At the same time, in order to solve the problem of inconsistent coordinate systems and scales of posture trajectories obtained by different sensors, this method uses the Umeyama algorithm / ICP algorithm to solve the transformation matrix and scale scaling coefficient between the two coordinate systems for the two posture trajectories in each motion state based on the hand-eye calibration principle, and aligns the two posture trajectories in space and scale. In this way, the positioning accuracy of the VR device can be accurately evaluated based on the two posture trajectories aligned in time, space, and scale, and the equipment cost is low and the operation is simple.

[0065] The evaluation method of the embodiment of the present application, on the one hand, compared with the evaluation method based on ultra-precision optical tracking systems such as ART, by building a Vicon motion capture system indoors, multiple infrared cameras can be used in a small space to accurately track VR devices with high frame rate and high resolution, and its cost is controllable, and the calibration of the multi-camera stereo imaging system and the establishment of the world coordinate system are relatively simple, and various complex data acquisition actions can be completed in a single-person operation. On the other hand, compared with the evaluation method using Outside-in spatial positioning monitoring technology, the Vicon motion capture system only needs to observe the photosensitive points (such as fluorescent balls) attached to the VR device to track the posture trajectory of the device, and output the trajectory parameters of the rigid body center of mass composed of the photosensitive points. The system complexity is low, and the photosensitive points can be easily reused on different VR devices, and the cost is low. On the other hand, compared with a single Vicon motion capture system, it can be combined with a robotic arm to obtain the posture of the VR device in a small range of motion space, realize multi-module evaluation, and expand the application scenario of the evaluation method.

[0066] like Figure 2As shown, the evaluation system for the posture trajectory of a VR device provided in the embodiment of the present application mainly includes two parts: a Vicon motion capture system and a robotic arm system. Among them, the Vicon motion capture system is composed of a plurality of fixed infrared cameras, which can be pre-calibrated with multiple cameras to determine the positional relationship between the cameras. The user can wear an HMD and hold a handle to perform a first movement within the field of view of multiple infrared cameras in the Vicon motion capture system. For example, the embodiment of the present application does not impose any restrictive requirements on the number of infrared cameras in the Vicon motion capture system. These cameras can be evenly fixed on a shelf or on a smooth wall, and can be adjusted according to the actual scene. The robotic arm system includes a checkerboard calibration plate and a movable robotic arm. The base of the robotic arm is fixed, and the HMD or handle can be fixed at the end of the robotic arm, and the second movement is performed as the robotic arm moves.

[0067] In an embodiment of the present application, in order to enable the Vicon motion capture system to observe the posture trajectory of the VR device, a plurality of reusable photosites are pasted on the surface of the VR device to mark the device itself. These photosites perform rigid body motion, and the rigid body center of mass formed is used to track the posture trajectory of the VR device.

[0068] Take the photosensitive point as a fluorescent ball as an example, Figure 3A and Figure 3B As shown, they are respectively rigid body effect diagrams of fluorescent balls pasted on the HMD and the handle, wherein the embodiment of the application does not impose any restrictive requirements on the number and material of the photosensing points, which can be adjusted according to actual conditions.

[0069] In the embodiment of the present application, although the robotic arm has a relatively small range of motion and a relatively simple posture, it has low cost and strong repeatability, while the Vicon motion capture system has a slightly higher cost and low repeatability, but its range of motion is larger and its posture is more flexible. The two have complementary advantages. Therefore, using the robotic arm system and the Vicon motion capture system to respectively evaluate the positioning accuracy of the VR device can be applied to different application requirements and has a wider range of applications.

[0070] It should be noted that when the target VR device is a handle, the checkerboard calibration plate in the robotic arm system can be omitted.

[0071] Based on the above-mentioned robotic arm system and Vicon motion capture system, the architecture diagram of the evaluation method of the VR device posture trajectory provided in the embodiment of the present application is as follows: Figure 4As shown in the figure, the pose trajectory of the target VR device during the movement is collected by the Vicon motion capture system and the robotic arm system, and the pose trajectory output by the target VR device itself is time-stamped, spatially and scale-aligned, and then the evaluation index between the pose trajectory collected by each system and the pose trajectory output by the target VR device itself is calculated, thereby completing the evaluation of the accuracy of the positioning and pose determination algorithm in the target VR device. Among them, when building the Vicon motion capture system, multiple infrared cameras are jointly calibrated to obtain the relative position relationship between the cameras, and the Vicon world coordinate system is established based on the relative position relationship. At the same time, multiple photosensitive points are evenly and asymmetrically pasted on the surface of the target VR device to establish a rigid body.

[0072] See also Figure 5 , is a flow chart of a method for evaluating a posture trajectory of a VR device provided in an embodiment of the present application, which mainly includes the following steps:

[0073] S50 1 obtains the first posture trajectory of the rigid body center of mass formed by multiple photosensing points collected by the Vicon motion capture system in the first motion state and the second posture trajectory of the target VR device itself, and obtains the third posture trajectory of the target VR device collected by the robotic arm system in the second motion state and the fourth motion trajectory of the target VR device itself.

[0074] In one example, in order to enable the Vicon motion capture system to track the pose trajectory of the target VR device, multiple photosites as rigid bodies are attached to the target VR device, and each infrared camera can obtain enough image frames to fully capture the evaluation features, and minimize the obstructions and fluorescent objects in the environment to reduce unnecessary interference. The photosites on the target VR device cannot block the camera and other sensors on the HMD. In this way, when the target VR device performs the first movement within the field of view of multiple infrared cameras in the Vicon motion capture system, multiple infrared cameras can collect photosite images, and by inputting these photosite images into a computer with image processing capabilities, the first pose trajectory of the rigid body center of mass on the target VR device is obtained. At the same time, the target VR device uses the data collected by its own sensors and uses VI-SLAM technology to output its second pose trajectory.

[0075] It should be noted that the light-sensitive spots on the target VR device are pasted on, so they can be removed and reused on different devices.

[0076] In the Vicon motion capture system, the target VR device has a large range of motion and a more flexible posture, which is suitable for evaluating the performance of the positioning and attitude algorithm during large-scale motion. In order to evaluate the performance of the positioning and attitude algorithm when the target VR device moves slightly, a robotic arm system is also used.

[0077] In one example, the target VR device is fixed at the end of the robotic arm and performs a second movement with the robotic arm. The motion trajectory of the end of the robotic arm relative to the base of the robotic arm is obtained through forward kinematics, and is used as the third posture trajectory of the target VR device collected by the robotic arm system. At the same time, while the target VR device moves with the robotic arm, an image of a checkerboard calibration plate can be captured to obtain its own fourth posture trajectory through the PnP algorithm.

[0078] In the embodiments of the present application, the Vicon motion capture system and the robotic arm system respectively include multiple coordinate systems, such as Fig. 6A and Figure 6B As shown, there are multiple sets of corresponding relationships between the coordinate systems in these two systems. Specifically, Fig. 6A The first reference coordinate system (i.e., SLAM world coordinate system or ground coordinate system) in which the target VR device records its own second pose trajectory in the Vicon motion capture system corresponds to Figure 6B The second reference coordinate system (i.e., the calibration plate coordinate system) in which the target VR device in the middle robot arm system records its own fourth posture trajectory; Fig. 6A The coordinate system of the target VR device in the Vicon motion capture system (such as the camera coordinate system or the IMU coordinate system, or the coordinate system with a point on the device (such as the eyebrow center of the HMD or the center point of the IMU of the handle) as the origin) corresponds to Figure 6B The own coordinate system of the target VR device in the robot arm system; Fig. 6A The rigid body center of mass coordinate system formed by multiple photosensitive points on the target VR device in the Vicon motion capture system corresponds to Figure 6B Coordinate system of the end of the robot arm in the robot arm system; Fig. 6A The first world coordinate system (i.e., Vicon world coordinate system) of the first position trajectory of the rigid body mass center on the target VR device collected by the Vicon motion capture system corresponds to the machine Figure 6B The second world coordinate system (i.e., the base world coordinate system of the robotic arm) of the third posture trajectory of the target VR device (equivalent to the end of the robotic arm) collected by the robotic arm system.

[0079] Based on the correspondence between the above coordinate systems, when the performance of the positioning and attitude algorithm of the target VR device is evaluated using the Vicon motion capture system and the robotic arm system, there is also a correspondence between the pose trajectory collected by the Vicon motion capture system and the pose trajectory collected by the robotic arm system. Specifically, the first pose trajectory of the center of mass of the rigid body on the target VR device in the Vicon motion capture system in the first world coordinate system corresponds to the third pose trajectory of the end of the robotic arm where the target VR device is located in the robotic arm system in the second world coordinate system; the second pose trajectory of the target VR device in the Vicon motion capture system recorded by itself in the first reference coordinate system corresponds to the fourth pose trajectory of the target VR device in the robotic arm system in the second reference coordinate system.

[0080] In one example, the target VR device may be an HMD or a handle, and this embodiment of the present application does not impose any restrictive requirements.

[0081] S502: For the two posture trajectories in each motion state, the angular velocity modulus is calculated at preset time intervals to obtain two angular velocity modulus sequences, and the timestamps of the two posture trajectories are aligned according to the two angular velocity modulus sequences.

[0082] In some scientific experiments, computer time (i.e., Unix timestamp) is usually used to record data, that is, the number of seconds that have passed since midnight on January 1, 1970, UTC time, excluding leap seconds. Computer time can unify data collected by different devices or time periods into the same time frame, which is convenient for comparison. For observation equipment without precise clocks, it is usually necessary to connect to the Internet to obtain the current computer time to correct its own clock drift. However, in indoor scenes such as laboratories, living rooms, and bedrooms, computers or mobile target VR devices are usually not connected to the Internet. Therefore, it is difficult to obtain unified computer time, and posture data are recorded in their own time system. In this case, when using the Vicon motion capture system and the robotic arm system to evaluate the positioning accuracy of the target VR device, the posture trajectories obtained by different devices need to be compared to align the timestamps.

[0083] In an embodiment of the present application, when the posture trajectory of the target VR device is evaluated respectively using the Vicon motion capture system and the robotic arm system, the second posture trajectory of the two posture trajectories in the first motion state and the fourth posture trajectory of the two posture trajectories in the second motion state can be recorded as trajectories to be evaluated, and the first posture trajectory of the two posture trajectories in the first motion state and the third posture trajectory in the second motion state can be recorded as true value trajectories. By comparing the trajectory to be evaluated with the true value trajectory in each motion state, the accuracy of the positioning and posture determination algorithm in the target VR device can be evaluated.

[0084] like Figure 7 As shown in the figure, it is a schematic diagram of the principle of aligning the timestamps of the trajectory to be evaluated and the true value trajectory, where T G is the observation time period of the trajectory to be evaluated output by the target VR device in the reference coordinate system, T B is the observation time period of the true value trajectory output by the system in the world coordinate system, S G is the trajectory to be evaluated, S B is the true value trajectory, S G and S B There are differences in time, space and scale. Timestamp alignment can be regarded as calculating the starting time difference ΔT between the trajectory to be evaluated and the true trajectory. GB The process. The trajectory S to be evaluated G and the truth trajectory S B It can be seen that the arcs traveled by the two posture trajectories in the same time period are the same, that is, l G = l B , it can also be understood that the angular velocity modulus at the same time is the same, that is, ||ω B ||=||ω G ||, therefore, the two pose trajectories can be divided into angular velocity modulus length sequences for timestamp alignment.

[0085] It should be noted that during the timestamp alignment process, it is necessary to ensure that the observation time period of the trajectory to be evaluated is a subset of the observation time period of the true value trajectory, that is, T B Contains T G The start time of the .

[0086] Taking the trajectory to be evaluated and the true trajectory in a motion state as an example, the timestamp alignment process can be seen in Figure 8 , mainly includes the following steps:

[0087] S5021: Calculate the angular velocity modulus of the trajectory to be evaluated at preset time intervals to obtain a first angular velocity modulus sequence, and calculate the angular velocity modulus of the true value trajectory at preset time intervals to obtain a second angular velocity modulus sequence.

[0088] In practical applications, the length of the trajectory to be evaluated is generally shorter than the length of the true value trajectory. Therefore, when calculating the angular velocity modulus at a preset time interval, the length of the obtained first angular velocity modulus sequence is shorter than the length of the obtained second angular velocity modulus sequence.

[0089] S5022: Slide the first angular velocity modulus length sequence on the second angular velocity modulus length sequence.

[0090] Since the length of the first angular velocity modulus length sequence is shorter than that of the second angular velocity modulus length sequence, the shorter first angular velocity modulus length sequence can be slid on the longer second angular velocity modulus length sequence.

[0091] S5023: After each sliding, calculate the Pearson correlation coefficient between the first angular velocity modulus length sequence and the second angular velocity modulus length sequence.

[0092] After each sliding, the Pearson correlation coefficient between the two angular velocity modulus length sequences is obtained by calculating the covariance and standard deviation of the first angular velocity modulus length sequence and the second angular velocity modulus length sequence within a certain search interval. This coefficient can reflect the linear correlation between the first angular velocity modulus length sequence and the second angular velocity modulus length sequence.

[0093] S5024: When the Pearson correlation coefficient is the largest, determine the starting time difference between the two posture trajectories according to the first angular velocity modulus length sequence and the second angular velocity modulus length sequence.

[0094] Specifically, the calculation formula for the starting time difference is:

[0095]

[0096] Among them, {ω G} is the first angular velocity modulus sequence, {ω B}The second angular velocity modulus sequence, l(ω G ,ω B ) represents the Pearson correlation coefficient.

[0097] S5025: Align the timestamps of the two pose trajectories according to the starting time difference.

[0098] Through the starting time difference between the trajectory to be evaluated and the true trajectory, the trajectory to be evaluated and the true trajectory can be unified into the same time frame, thereby completing the alignment of timestamps.

[0099] In an embodiment of the present application, when the timestamps of two posture trajectories in each motion state are aligned, the angular velocity modulus can be calculated based on Lie algebra theory.

[0100] In one example, the pose in the trajectory to be estimated can represent the transformation matrix of the VR device's own coordinate system relative to the reference coordinate system, which is denoted as The pose in the true trajectory can represent the transformation matrix of the rigid body center of mass coordinate system or the robot end coordinate system relative to the world coordinate system, which is recorded as According to the continuous domain representation of Lie algebra, the transformation matrix T between the estimated trajectory and the true value trajectory can be expressed as:

[0101]

[0102]

[0103] Where R is the rotation matrix, t is the translation vector, ρ is the translation component of the Lie algebra, J is the left Jacobian matrix of SO(3) corresponding to the rotation matrix, and Φ is the rotation component of the Lie algebra.

[0104] Assuming that the rotation matrix and translation vector are expressed separately, the kinematic equation can be written as:

[0105]

[0106] Among them, ω represents the rotation velocity vector, v represents the translation velocity vector, and using the transformation matrix, ω and v can represent the generalized velocity, which is recorded as

[0107] By rotating the velocity vector ω, the angular velocity modulus ||ω|| can be obtained.

[0108] In practical applications, the velocity is usually calculated using the discrete domain of Lie algebra. In this case, the calculation process of the angular velocity modulus can be found in Fig. 9 , mainly includes the following steps:

[0109] S5021_1: According to the trajectory to be evaluated, obtain the estimated posture recorded by the target VR device itself at the first moment and the second moment, and, according to the true value trajectory, obtain the real posture observed by the system at the first moment and the second moment.

[0110] For any pose trajectory in the trajectory to be evaluated and the true value trajectory, the pose obtained at the first moment t1 and the second moment t2 are expressed by Lie groups as follows:

[0111]

[0112] S5021_2: Determine a first transformation matrix including rotation information and translation information based on the estimated postures at the first moment and the second moment, and determine a second transformation matrix including rotation information and translation information based on the actual postures at the first moment and the second moment.

[0113] For any posture trajectory in the trajectory to be evaluated and the true value trajectory, the calculation formula for the velocity in the time period from t1 to t2 is:

[0114]

[0115] Among them, T 1 -1 T 2 Represents the transformation matrix between two points, including 3D rotation information and 3D translation information.

[0116] S5021_3: Separately represent the rotation information and the translation information in the first transformation matrix, and obtain a first angular velocity vector by deriving the rotation information; and separately represent the rotation information and the translation information in the second transformation matrix, and obtain a second angular velocity vector by deriving the rotation information.

[0117] When Δt is small, it can be considered that:

[0118]

[0119] In one example, ΔT is a transformation matrix ΔT=[Δx Δt Δz Δtx Δty Δtz] that represents the 3D rotation information and the 3D translation information separately according to Lie algebraic transformation, where the linear velocity vector Angular velocity vector

[0120] S5021_4: Obtain a first angular velocity modulus according to the first angle vector, and obtain a second angular velocity modulus according to the second angular velocity vector.

[0121] According to the components of the angular velocity vector on the xyz axis, the angular velocity modulus in the time interval from t1 to t2 can be obtained.

[0122] S503: For the two posture trajectories with aligned timestamps in each motion state, a relative relationship between the two coordinate systems at different times is established, and the two posture trajectories are spatially aligned and scale-aligned according to the relative relationship.

[0123] The pose trajectories obtained based on the data collected by different sensors are all based on their own spatial coordinate systems, and the unit scales are often different. Therefore, when comparing and evaluating the trajectory to be evaluated and the true trajectory in each motion state, the two pose trajectories need to be transformed into the same coordinate system and have the same scale. Among them, for the coordinate system transformation, based on the hand-eye calibration principle, the Umeyama algorithm / ICP algorithm can be used to optimize and solve the relevant transformation matrix and scale scaling coefficient parameters, so as to achieve spatial alignment and scale alignment of the two pose trajectories in each motion state.

[0124] Eye-in-Hand calibration is usually used for coordinate system conversion and control of systems such as robotic arms. Since the modules in the Vicon motion capture system in the embodiment of the present application can be abstracted as modules of the robotic arm system, and there is a corresponding relationship between the coordinate systems of the modules, the hand-eye calibration principle can also be used to transform the spatial coordinate system of the Vicon motion capture system, thereby solving the spatial and scale differences between the trajectory to be evaluated and the true trajectory.

[0125] In hand-eye calibration, the target VR device is fixed to the end of the robot arm and moves with the movement of the robot arm. Therefore, the coordinate system of the target VR device itself is fixed relative to the coordinate system of the end of the robot arm, while the coordinate system of the target VR device itself is variable relative to the coordinate system of the robot arm base (i.e., the second world coordinate system). Similarly, in the Vicon motion capture system, the coordinate system of the rigid body center of mass is fixed relative to the coordinate system of the target VR device itself, while the coordinate system of the target VR device itself is variable relative to the Vicon world coordinate system (i.e., the first world coordinate system).

[0126] In one example, when the third pose trajectory of the end of the manipulator (i.e., the target VR device) in the second world coordinate system is known, and the fourth pose trajectory of the target VR device itself in the second reference coordinate system (i.e., the calibration plate coordinate system) is recorded, due to the transformation matrix of the coordinate system of the target VR device itself relative to the coordinate system of the end of the manipulator is fixed. At this time, once the transformation matrix of the target VR device's own coordinate system relative to the robotic arm's end coordinate system is obtained The transformation matrix between the second world coordinate system and the calibration plate coordinate system can be obtained Similarly, when the first pose trajectory of the rigid body mass center in the first world coordinate system is known, and the second pose trajectory of the target VR device itself recorded in the first reference coordinate system (i.e., the ground coordinate system), due to the transformation matrix of the rigid body mass center coordinate system relative to the coordinate system of the target VR device itself is fixed. At this time, once the transformation matrix of the rigid body center of mass coordinate system relative to the target VR device's own coordinate system is obtained The transformation matrix between the first world coordinate system and the ground coordinate system can be obtained

[0127] In an embodiment of the present application, the first pose trajectory can represent the transformation matrix of the rigid body center of mass coordinate system relative to the first world coordinate system: The second pose trajectory can represent the transformation matrix of the target VR device's own coordinate system relative to the first reference coordinate system Similarly, the third pose trajectory can represent the transformation matrix of the end of the robot arm relative to the second world coordinate system The fourth pose trajectory can represent the transformation matrix of the target VR device's own coordinate system relative to the second reference coordinate system

[0128] When the first pose trajectory and the third pose estimation are recorded as the true value trajectory, and the second pose trajectory and the fourth pose trajectory are recorded as the trajectory to be evaluated, the spatial and scale alignment process of the trajectory to be estimated and the true value trajectory under each motion state can be found in Fig.10 , mainly includes the following steps:

[0129] S5031: For the estimated trajectory and the true trajectory with aligned timestamps under each motion state, a relative relationship between the reference coordinate system of the estimated trajectory and the world coordinate system of the true trajectory is established according to the 3D points on the estimated trajectory and the projection points of the 3D points on the true trajectory.

[0130] When the timestamps of the estimated trajectory and the true trajectory are aligned in each motion state, according to the projection rule, the following projection relationship exists:

[0131]

[0132] Among them, P G is a 3D point on the trajectory to be evaluated, P B is the projection point corresponding to the 3D point on the true value trajectory, through The transformation matrix between the reference coordinate system where the estimated trajectory is located and the world coordinate system where the true value trajectory is located can be obtained Among them, when the motion state is the first motion state, P G is the 3D coordinate in the ground coordinate system (i.e., the first reference coordinate system), P B is the 3D coordinate in the first world coordinate system, Represents the transformation matrix between the rigid body center of mass coordinate system and the first world coordinate system, Represents the transformation matrix between the rigid body center of mass coordinate system and the target VR device's own coordinate system. represents the transformation matrix between the coordinate system of the target VR device itself and the ground coordinate system; when the motion state is the second motion state, P G is the 3D coordinate in the calibration plate coordinate system (i.e., the second reference coordinate system), P B is the 3D coordinate in the second world coordinate system, Represents the transformation matrix between the robot end coordinate system and the second world coordinate system, Represents the transformation matrix between the coordinate system of the end of the robot arm and the coordinate system of the target VR device itself. Represents the transformation matrix between the coordinate system of the target VR device and the coordinate system of the calibration board.

[0133] S5032: Establish a transformation matrix equation based on the invariance principle of the relative relationship between two coordinate systems at different times.

[0134] For the estimated trajectory and the true trajectory in each motion state, for the two observations before and after the same point, we can get:

[0135]

[0136] Among them, formula 8 can be understood as the relative relationship between the two coordinate systems at different times is fixed, that is, After transposition transformation, we can get:

[0137]

[0138] For Formula 9, let Then the transformation matrix equation can be established as:

[0139] AX=XB Formula 10

[0140] Therefore, the hand-eye calibration problem of the trajectory to be evaluated and the true trajectory in each motion state can be summarized as the problem of solving X in the above formula 10.

[0141] S5033: According to two points on the trajectory to be evaluated and the true trajectory at the same time, a transformation matrix between the coordinate system where the trajectory to be evaluated is located and the coordinate system where the true trajectory is located, and a scale scaling coefficient between the trajectory to be evaluated and the true trajectory are solved.

[0142] In one example, the solution of the transformation matrix equation can be converted into the least squares iterative optimization solution through Lie group / Lie algebra theory. At the same time, according to the two points on the trajectory to be evaluated and the true value trajectory at the same time, the transformation matrix between the rigid body center of mass coordinate system and the target VR device's own coordinate system is optimized and solved using the Umeyama algorithm or the ICP algorithm. and scale factor, or the transformation matrix between the coordinate system of the end-arm and the coordinate system of the target VR device itself And the scale scaling factor, further, according to the projection relationship shown in formula 1, the transformation matrix between the reference coordinate system and the world coordinate system can be solved

[0143] S5034: spatially aligning the trajectory to be evaluated with the true trajectory according to the transformation matrix, and scale-aligning the trajectory to be evaluated with the true trajectory according to the scale scaling coefficient.

[0144] The spatial and scale differences between the timestamp-aligned trajectories to be evaluated and the true trajectories can be resolved through the transformation matrix and scale scaling coefficients, thus enabling comparative evaluation.

[0145] S504: For the two pose trajectories aligned in time stamp, space and scale in each motion state, calculate the difference between the two pose trajectories.

[0146] For the two pose trajectories after time stamp, space and scale alignment in each motion state, the difference between the three-dimensional coordinates and three-axis rotation angles of the two pose trajectories at the same time can be calculated.

[0147] Optionally, the evaluation index of the difference between two pose trajectories includes at least one of the root mean square error, error mean, mean error, standard deviation, minimum error and maximum error of the absolute trajectory error (ATE).

[0148] S505: Comprehensively evaluate the positioning accuracy of the target VR device based on each group of differences.

[0149] Since the Vicon motion capture system supports large-scale movement of the target VR device within the field of view of multiple infrared cameras, the system's evaluation difference on the position and trajectory of the target VR device is applicable to the state where the target VR device moves in a large range. The movement range of the robotic arm in the robotic arm system is small. Therefore, the system's evaluation difference on the position and trajectory of the target VR device is applicable to the state where the target VR device moves in a small range. The combination of the two can be used for evaluation under different motion states, with a wider range and higher accuracy.

[0150] In the embodiments of the present application, in order to accurately track and evaluate the accuracy of spatial parameters such as the position and posture of the 6DoF VR device in an environment with a small activity space (such as indoors), a single-person-operated Vicon motion capture system and a robotic arm system are built to obtain the precise posture trajectory of the VR device, and compare them with the posture trajectory output by the VR device itself, so as to comprehensively evaluate the performance of the positioning and posture determination algorithm in the VR device. The equipment cost is low and the operation is simple. In addition, since the VR device in the robotic arm system has a small motion range and a single posture, while the VR device in the Vicon motion capture system has a large motion range and a flexible posture, the two systems are used to evaluate the VR device separately, which can meet the usage requirements of the VR device in different scenarios and has a wider range of applications. Among them, during the evaluation process, in order to solve the problem of inconsistent timestamps of posture trajectories obtained by different sensors, for the two posture trajectories under each system, the angular velocity modulus is obtained at a preset time interval based on the Lie algebra theory, and the starting time difference of the two angular velocity modulus sequences is obtained by calculating the Pearson correlation coefficient, so as to align the two posture trajectories in time. At the same time, in order to solve the problem of inconsistent coordinate systems and scales of posture trajectories obtained by different sensors, for the two posture trajectories under each system, based on the hand-eye calibration principle, the Umeyama algorithm / ICP algorithm is used to solve the transformation matrix and scale scaling coefficient between the two coordinate systems, so as to align the two posture trajectories in space and scale. In this way, based on the two posture trajectories aligned in time, space and scale, the positioning accuracy of VR equipment can be accurately evaluated, providing a reference for the design and safe use of related products.

[0151] In one example, when the positioning and attitude accuracy of the target VR device is evaluated using the Vicon motion capture system and the robotic arm system, in terms of algorithm program design, a corresponding interface function is written according to the characteristics of the Vicon motion capture system, the robotic arm system, and the target VR device data stream. By calling the corresponding interface function, the corresponding posture trajectory can be obtained, which is convenient for expansion. At the same time, the storage path of the posture trajectory obtained by the interface function is written into the same configuration file. In this way, in actual applications, by modifying the parameters in the configuration file, different systems can be easily switched to evaluate the positioning and attitude accuracy of the target VR device. That is, the two systems share a set of software programs for evaluation, and the implementation process is simple and efficient.

[0152] In one example, the output paths of the evaluation results of the positioning and posture accuracy of the target VR device using the Vicon motion capture system and the robotic arm system respectively can also be written into the same configuration file, thereby facilitating the comprehensive analysis of different evaluation results.

[0153] Based on the same technical concept, an embodiment of the present application provides an evaluation device, which can be a server, including but not limited to a cloud server, a server cluster, a single server, etc., or a client, including but not limited to a laptop computer, a desktop computer, etc., and can implement the steps of the above-mentioned VR device posture trajectory evaluation method and achieve the same technical effect.

[0154] See also Fig.11 , the processor 1101, the memory 1102 and the communication interface 1103, the communication interface 1103, the memory 1102 and the processor 1101 are connected via a bus 1104;

[0155] The memory 1102 stores a computer program, and the processor 1101 performs the following operations according to the computer program:

[0156] Through the communication interface 1103, a first pose trajectory of the rigid body center of mass formed by multiple photosensing points collected by the Vicon motion capture system in the first motion state and a second pose trajectory of the target VR device itself are obtained, and a third pose trajectory of the target VR device collected by the robotic arm system in the second motion state and a fourth motion trajectory of the target VR device itself are obtained; wherein, the Vicon motion capture system includes multiple fixed infrared cameras, and the target VR device with multiple photosensing points attached thereto performs a first motion within the field of view of the multiple infrared cameras, and the robotic arm system includes a checkerboard calibration plate and a movable robotic arm, and the target device is fixed to the end of the robotic arm and performs a second motion with the robotic arm;

[0157] For the two posture trajectories under each motion state, the angular velocity modulus is calculated at preset time intervals to obtain two angular velocity modulus sequences, and the timestamps of the two posture trajectories are aligned according to the two angular velocity modulus sequences;

[0158] For the two posture trajectories with aligned timestamps in each motion state, a relative relationship between the two coordinate systems at different times is established, and the two posture trajectories are spatially aligned and scale-aligned according to the relative relationship;

[0159] For each motion state, the difference between the two pose trajectories after the timestamps, space and scale are aligned is calculated;

[0160] The positioning accuracy of the target VR device is comprehensively evaluated based on each group of differences.

[0161] Optionally, the processor 1101 aligns the timestamps of the two posture trajectories according to the two angular velocity module length sequences, and the specific operation is:

[0162] Sliding a first angular velocity modulus length sequence on a second angular velocity modulus length sequence, wherein the length of the first angular velocity modulus length sequence is shorter than the length of the second angular velocity modulus length sequence;

[0163] After each sliding, calculating the Pearson correlation coefficient between the first angular velocity modulus length sequence and the second angular velocity modulus length sequence;

[0164] When the Pearson correlation coefficient is the largest, determining the starting time difference between two posture trajectories according to the first angular velocity modulus length sequence and the second angular velocity modulus length sequence;

[0165] According to the starting time difference, the timestamps of the two pose trajectories are aligned.

[0166] Optionally, the second posture trajectory of the two posture trajectories in the first motion state and the fourth posture trajectory of the two posture trajectories in the second motion state are recorded as trajectories to be evaluated, the first posture trajectory of the two posture trajectories in the first motion state and the third posture trajectory in the second motion state are recorded as true value trajectories, and the processor 1101 calculates the angular velocity modulus in the following manner:

[0167] According to the trajectory to be evaluated, obtaining the estimated posture recorded by the target VR device itself at the first moment and the second moment, and, according to the true value trajectory, obtaining the real posture observed by the system at the first moment and the second moment;

[0168] Determine a first transformation matrix including rotation information and translation information according to the estimated postures at the first moment and the second moment, and determine a second transformation matrix including rotation information and translation information according to the actual postures at the first moment and the second moment;

[0169] The rotation information and the translation information in the first transformation matrix are represented separately, and a first angular velocity vector is obtained by deriving the rotation information, and the rotation information and the translation information in the second transformation matrix are represented separately, and a second angular velocity vector is obtained by deriving the rotation information;

[0170] A first angular velocity modulus is obtained according to the first angle vector, and a second angular velocity modulus is obtained according to the second angular velocity vector.

[0171] Optionally, the coordinate systems in the Vicon motion capture system and the robotic arm system have the following corresponding relationship:

[0172] The first reference coordinate system in which the target VR device in the Vicon motion capture system records its own second posture trajectory corresponds to the second reference coordinate system in which the target VR device in the robotic arm system records its own fourth posture trajectory;

[0173] The first world coordinate system of the first pose trajectory of the rigid body center of mass formed by the multiple photosensing points acquired by the Vicon motion capture system corresponds to the second world coordinate system of the third pose trajectory of the end of the robotic arm acquired by the robotic arm system;

[0174] The local coordinate system of the target VR device in the Vicon motion capture system corresponds to the local coordinate system of the target VR device in the robotic arm system;

[0175] The rigid body centroid coordinate system formed by multiple photosensing points on the target VR device in the Vicon motion capture system corresponds to the robotic arm end coordinate system in the robotic arm system.

[0176] Optionally, the second posture trajectory of the two posture trajectories in the first motion state and the fourth posture trajectory of the two posture trajectories in the second motion state are recorded as trajectories to be evaluated, and the first posture trajectory of the two posture trajectories in the first motion state and the third posture trajectory in the second motion state are recorded as true value trajectories;

[0177] The processor 1101 establishes a relative relationship between two coordinate systems at different times for two posture trajectories with time stamps aligned in each motion state, and performs spatial alignment and scale alignment on the two posture trajectories according to the relative relationship. The specific operations are as follows:

[0178] For the estimated trajectory and the true trajectory with aligned timestamps under each motion state, a relative relationship between the reference coordinate system of the estimated trajectory and the world coordinate system of the true trajectory is established according to the 3D points on the estimated trajectory and the projection points of the 3D points on the true trajectory;

[0179] According to the invariance principle of the relative relationship between two coordinate systems at different times, the transformation matrix equation is established;

[0180] According to two points on the trajectory to be evaluated and the true value trajectory at the same time, solving the transformation matrix between the coordinate system where the trajectory to be evaluated is located and the coordinate system where the true value trajectory is located, and the scale scaling coefficient between the trajectory to be evaluated and the true value trajectory;

[0181] The trajectory to be evaluated and the true trajectory are spatially aligned according to the transformation matrix, and the trajectory to be evaluated and the true trajectory are scale-aligned according to the scale scaling coefficient.

[0182] Optionally, the Vicon motion capture system, the robotic arm system, and the target VR device each correspond to an interface function, and the interface function is used to obtain and store corresponding posture trajectories, and the storage path of each posture trajectory is written into the same configuration file, so that the Vicon motion capture system and the robotic arm system use the same set of software programs to evaluate the positioning accuracy of the target VR device according to different parameters in the configuration file.

[0183] Optionally, the evaluation index of the difference between the two posture trajectories includes at least one of the root mean square error, error mean, mean error, standard deviation, minimum error and maximum error of the absolute trajectory error.

[0184] Optionally, the target VR device is any one of a head-mounted display HMD and a handle.

[0185] It should be noted that Fig.11 This is just an example, and provides the necessary hardware for the evaluation device to execute the steps of the evaluation method for the posture trajectory of a VR device provided in the embodiment of the present application. If not shown, the evaluation device may also include the hardware of conventional data processing devices such as a keyboard, a mouse, and a power supply.

[0186] in, Fig.11The memory 1102 in the embodiment may be a volatile memory, such as a random-access memory (RAM); the memory 1102 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1102 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be a combination of the above memories; the processor 1101 may include one or more central processing units (CPUs), graphics processors or digital processing units, etc.

[0187] An embodiment of the present application also provides a computer-readable storage medium for storing some instructions, which, when executed, can complete a method for evaluating the posture trajectory of a VR device in the aforementioned embodiment.

[0188] An embodiment of the present application also provides a computer program product for storing a computer program, wherein the computer program is used to execute a method for evaluating a posture trajectory of a VR device in the aforementioned embodiment.

[0189] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0190] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0191] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0192] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0193] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for evaluating the posture trajectory of a VR device, characterized in that: The method comprises a Vicon motion capture system and a robotic arm system, wherein the Vicon motion capture system comprises a plurality of fixed infrared cameras, a target VR device with a plurality of light-sensitive points attached thereto performs a first movement within the field of view of the plurality of infrared cameras, the robotic arm system comprises a checkerboard calibration plate and a movable robotic arm, the target device is fixed to the end of the robotic arm and performs a second movement with the robotic arm, and the method comprises: Obtaining a first pose trajectory of a rigid body centroid formed by a plurality of light-sensitive points acquired by the Vicon motion capture system in a first motion state and a second pose trajectory of the target VR device itself, and obtaining a third pose trajectory of the target VR device acquired by the manipulator system in a second motion state and a fourth motion trajectory of the target VR device itself; For the two posture trajectories under each motion state, the angular velocity modulus is calculated at preset time intervals to obtain two angular velocity modulus sequences, and the timestamps of the two posture trajectories are aligned according to the two angular velocity modulus sequences; For the two posture trajectories with aligned timestamps in each motion state, a relative relationship between the two coordinate systems at different times is established, and the two posture trajectories are spatially aligned and scale-aligned according to the relative relationship; For each motion state, the difference between the two pose trajectories after the timestamps, space and scale are aligned is calculated; The positioning accuracy of the target VR device is comprehensively evaluated based on each group of differences.

2. The method according to claim 1, characterized in that The step of aligning the timestamps of the two pose trajectories according to the two angular velocity modulus length sequences includes: Sliding a first angular velocity modulus length sequence on a second angular velocity modulus length sequence, wherein the length of the first angular velocity modulus length sequence is shorter than the length of the second angular velocity modulus length sequence; After each sliding, calculating the Pearson correlation coefficient between the first angular velocity modulus length sequence and the second angular velocity modulus length sequence; When the Pearson correlation coefficient is the largest, determining the starting time difference between two posture trajectories according to the first angular velocity modulus length sequence and the second angular velocity modulus length sequence; According to the starting time difference, the timestamps of the two pose trajectories are aligned.

3. The method according to claim 2, characterized in that The second posture trajectory of the two posture trajectories under the first motion state and the fourth posture trajectory of the two posture trajectories under the second motion state are recorded as the trajectory to be evaluated, the first posture trajectory of the two posture trajectories under the first motion state and the third posture trajectory under the second motion state are recorded as the true value trajectory, and the angular velocity modulus is calculated by the following method: According to the trajectory to be evaluated, obtaining the estimated posture recorded by the target VR device itself at the first moment and the second moment, and, according to the true value trajectory, obtaining the real posture observed by the system at the first moment and the second moment; Determine a first transformation matrix including rotation information and translation information according to the estimated postures at the first moment and the second moment, and determine a second transformation matrix including rotation information and translation information according to the actual postures at the first moment and the second moment; The rotation information and the translation information in the first transformation matrix are represented separately, and a first angular velocity vector is obtained by deriving the rotation information, and the rotation information and the translation information in the second transformation matrix are represented separately, and a second angular velocity vector is obtained by deriving the rotation information; A first angular velocity modulus is obtained according to the first angle vector, and a second angular velocity modulus is obtained according to the second angular velocity vector.

4. The method according to claim 1, characterized in that The coordinate systems in the Vicon motion capture system and the robotic arm system have the following corresponding relationship: The first reference coordinate system in which the target VR device in the Vicon motion capture system records its own second posture trajectory corresponds to the second reference coordinate system in which the target VR device in the robotic arm system records its own fourth posture trajectory; The first world coordinate system of the first pose trajectory of the rigid body center of mass formed by the multiple photosensing points acquired by the Vicon motion capture system corresponds to the second world coordinate system of the third pose trajectory of the end of the robotic arm acquired by the robotic arm system; The local coordinate system of the target VR device in the Vicon motion capture system corresponds to the local coordinate system of the target VR device in the robotic arm system; The rigid body centroid coordinate system formed by multiple photosensing points on the target VR device in the Vicon motion capture system corresponds to the robotic arm end coordinate system in the robotic arm system.

5. The method according to claim 4, characterized in that The second posture trajectory of the two posture trajectories in the first motion state and the fourth posture trajectory of the two posture trajectories in the second motion state are recorded as trajectories to be evaluated, and the first posture trajectory of the two posture trajectories in the first motion state and the third posture trajectory in the second motion state are recorded as true value trajectories; The method of establishing a relative relationship between two coordinate systems at different times for the two posture trajectories with time stamps aligned under each motion state, and performing spatial alignment and scale alignment on the two posture trajectories according to the relative relationship includes: For the estimated trajectory and the true trajectory with aligned timestamps under each motion state, a relative relationship between the reference coordinate system of the estimated trajectory and the world coordinate system of the true trajectory is established according to the 3D points on the estimated trajectory and the projection points of the 3D points on the true trajectory; According to the invariance principle of the relative relationship between two coordinate systems at different times, the transformation matrix equation is established; According to two points on the trajectory to be evaluated and the true value trajectory at the same time, solving the transformation matrix between the coordinate system where the trajectory to be evaluated is located and the coordinate system where the true value trajectory is located, and the scale scaling coefficient between the trajectory to be evaluated and the true value trajectory; The trajectory to be evaluated and the true trajectory are spatially aligned according to the transformation matrix, and the trajectory to be evaluated and the true trajectory are scale-aligned according to the scale scaling coefficient.

6. The method according to any one of claims 1 to 5, characterized in that The Vicon motion capture system, the robotic arm system, and the target VR device each correspond to an interface function, and the interface function is used to obtain and store corresponding posture trajectories, and the storage path of each posture trajectory is written into the same configuration file, so that the Vicon motion capture system and the robotic arm system can evaluate the positioning accuracy of the target VR device respectively according to different parameters in the configuration file using the same set of software programs.

7. The method according to any one of claims 1 to 5, characterized in that The evaluation index of the difference between the two posture trajectories includes at least one of the root mean square error, error mean, mean error, standard deviation, minimum error and maximum error of the absolute trajectory error.

8. The method according to any one of claims 1 to 5, characterized in that The target VR device is any one of a head mounted display HMD and a handle.

9. An evaluation device, characterized in that It includes a processor, a memory and a communication interface, wherein the communication interface, the memory and the processor are connected via a bus; The memory stores a computer program, and the processor performs the following operations according to the computer program: Through the communication interface, a first pose trajectory of the rigid body center of mass formed by multiple photosensing points collected by the Vicon motion capture system in the first motion state and a second pose trajectory of the target VR device itself are obtained, and a third pose trajectory of the target VR device collected by the robotic arm system in the second motion state and a fourth motion trajectory of the target VR device itself are obtained; wherein, the Vicon motion capture system includes multiple fixed infrared cameras, and the target VR device with multiple photosensing points attached thereto performs a first motion within the field of view of the multiple infrared cameras, and the robotic arm system includes a checkerboard calibration plate and a movable robotic arm, and the target device is fixed to the end of the robotic arm and performs a second motion with the robotic arm; For the two posture trajectories under each motion state, the angular velocity modulus is calculated at preset time intervals to obtain two angular velocity modulus sequences, and the timestamps of the two posture trajectories are aligned according to the two angular velocity modulus sequences; For the two posture trajectories with aligned timestamps in each motion state, a relative relationship between the two coordinate systems at different times is established, and the two posture trajectories are spatially aligned and scale-aligned according to the relative relationship; For each motion state, the difference between the two pose trajectories after the timestamps, space and scale are aligned is calculated; The positioning accuracy of the target VR device is comprehensively evaluated based on each group of differences.

10. The evaluation device according to claim 9, characterized in that The Vicon motion capture system, the robotic arm system, and the target VR device each correspond to an interface function, and the interface function is used to obtain and store corresponding posture trajectories, and the storage path of each posture trajectory is written into the same configuration file, so that the Vicon motion capture system and the robotic arm system can evaluate the positioning accuracy of the target VR device respectively according to different parameters in the configuration file using the same set of software programs.

Images