Virtual reality tracker and method of tracker correction position

Abstract

A virtual reality tracker includes a first part and a second part. The first part includes a plurality of first light-emitting diodes (LEDs) and an inertial measurement unit (IMU). The IMU is configured to measure an acceleration and a three-axis angular velocity of the first part. The second part includes a plurality of second LEDs. The first part and the second part are connected by a flexible member.

Description

Technical Field

[0001] This invention relates to virtual reality systems, and more particularly to a virtual reality tracker and a method for calibrating the position of the tracker suitable for virtual reality systems. Background Technology

[0002] Generally, users need to wear a head-mounted display to experience scenes in virtual reality and interact with virtual objects in the scene through controllers or trackers. The number of light-emitting diodes (LEDs) on the controllers or trackers needs to be sufficient for the head-mounted display to capture images of these LEDs through cameras in order to track the position of the controllers or trackers in three-dimensional space.

[0003] However, when the number of LEDs on the controller or tracker reaches a certain level, the wearable tracker may become too large, even larger than the surface of a typical watch. This can prevent the wearable tracker from being properly secured to the user's body. For example, if the tracker is too large, it may not fit properly and conform to the user's wrist. When an oversized tracker is secured to the wrist with a strap, there may be a gap between the tracker and the wrist, causing the head-mounted display to be unable to accurately track the position of the controller or tracker in three-dimensional space.

[0004] Therefore, how to enable head-mounted display devices to accurately track the position of wearable trackers in three-dimensional space has become one of the problems to be solved in this field. Summary of the Invention

[0005] This invention provides a virtual reality tracker, which includes a first part and a second part. The first part includes a plurality of first light-emitting diodes (LEDs) and an inertial measurement unit (IMU). The IMU is used to measure an acceleration and a triaxial angular velocity of the first part. The second part includes a plurality of second LEDs. The first part and the second part are connected by a flexible component.

[0006] In one embodiment, the inertial measurement unit includes an accelerometer and a gyroscope. The accelerometer measures the acceleration. The gyroscope measures the three-axis angular velocity. The inertial measurement unit transmits the acceleration and the three-axis angular velocity to an inertial calculation program via a transmission interface. The inertial calculation program calculates a detailed attitude using an image attitude backtesting method based on the acceleration and the three-axis angular velocity. A head-mounted display (HMD) generates a real-time image based on this detailed attitude.

[0007] In one embodiment, the inertial measurement unit transmits the acceleration and the triaxial angular velocity to a head-mounted display device via a transmission interface; a first camera on the head-mounted display device captures images of the plurality of first light-emitting diodes on the first component to generate a plurality of first images; and the head-mounted display device calculates a first pose of the plurality of first light-emitting diodes in a three-dimensional space using the plurality of first images, the acceleration, and the triaxial angular velocity.

[0008] In one embodiment, the head-mounted display device transmits a first attitude to an inertial calculation program in the program. The inertial measurement unit transmits acceleration and triaxial angular velocity to the inertial calculation program through a transmission interface. The inertial calculation program calculates the detailed attitude based on the first attitude, acceleration, and triaxial angular velocity using the image attitude backtesting integration method. The head-mounted display device generates a real-time image based on the detailed attitude.

[0009] In one embodiment, the first camera or a second camera on the head-mounted display device captures images of the plurality of second light-emitting diodes on the second component to generate a plurality of second images; and the head-mounted display device uses the plurality of second images to calculate a second pose of the plurality of second light-emitting diodes in the three-dimensional space.

[0010] In one embodiment, a first coordinate system of the first pose corresponds to the first camera, and a second coordinate system of the second pose corresponds to the second camera, wherein the first coordinate system and the second coordinate system are different.

[0011] In one embodiment, the first camera and the second camera are located on the same head-mounted display device. The distance conversion between the first camera and the second camera is defined when the head-mounted display device is constructed. Therefore, the conversion between the first coordinate system and the second coordinate system is a known universal coordinate system. All attitude calculations are based on the universal coordinate system.

[0012] In one embodiment, the flexible component is a strap. When the first component and the second component are strapped to an object, the first posture of the first component at an initial time multiplied by a displacement is considered equal to the first posture of the first component at the first time. And the second posture of the second component at the initial time multiplied by the displacement, and then multiplied by a transpose function, will result in a calculation that is equal to the first posture of the first component at the first time.

[0013] This invention provides a tracker position correction method applicable to virtual reality systems. The tracker position correction method includes: capturing images of multiple first light-emitting diodes (LEDs) on a first part to obtain multiple first images; measuring an acceleration and a three-axis angular velocity of the first part using an inertial measurement unit (IMU); capturing images of multiple second LEDs on a second part to obtain multiple second images; and correcting the position of a tracker based on the multiple first images, the acceleration, the three-axis angular velocity, and the multiple second images; wherein the first part and the second part are connected by a flexible component.

[0014] In one embodiment, an accelerometer of the inertial measurement unit is used to measure the acceleration; and a gyroscope of the inertial measurement unit is used to measure the three-axis angular velocity; wherein, the inertial calculation program calculates a detailed attitude based on the acceleration and the three-axis angular velocity using an image interpolation method, and transmits the detailed attitude to a head-mounted display (HMD) through a transmission interface, and the head-mounted display generates a real-time image based on the detailed attitude.

[0015] In one embodiment, the inertial measurement unit transmits the acceleration and the triaxial angular velocity to a head-mounted display device via a transmission interface. A first camera on the head-mounted display device captures images of the plurality of first light-emitting diodes on the first component to generate the plurality of first images. The head-mounted display device calculates a first pose of the plurality of first light-emitting diodes in a three-dimensional space using the plurality of first images, the acceleration, and the triaxial angular velocity.

[0016] In one embodiment, the head-mounted display device transmits the first attitude back to the inertial calculation program. The inertial calculation program calculates a detailed attitude based on the first attitude, the acceleration, and the three-axis angular velocity using an image interpolation method, and transmits the detailed attitude to the head-mounted display device through a transmission interface. The head-mounted display device generates a real-time image based on the detailed attitude.

[0017] In one embodiment, the first camera or a second camera on the head-mounted display device captures images of the plurality of second light-emitting diodes on the second component to generate the plurality of second images. The head-mounted display device then uses these images to calculate a second orientation of the plurality of second light-emitting diodes in the three-dimensional space.

[0018] In one embodiment, a first coordinate system of the first pose corresponds to the first camera, and a second coordinate system of the second pose corresponds to the second camera, wherein the first coordinate system and the second coordinate system are different.

[0019] In one embodiment, the first camera and the second camera are located on the same head-mounted display device. The distance conversion between the first camera and the second camera is defined when the head-mounted display device is constructed. Therefore, the conversion between the first coordinate system and the second coordinate system is a known universal coordinate system. All attitude calculations are based on the universal coordinate system.

[0020] In one embodiment, the flexible component is a strap. After the first component and the second component are strapped to an object, the tracker position correction method further includes: multiplying the first posture of the first component at an initial time by a rotational displacement, which is considered equal to the first posture of the first component at the first time; and multiplying the second posture of the second component at the initial time by the rotational displacement, which is considered equal to the second posture of the second component at the first time, and then multiplying the second posture by a transpose function, the calculated result will be equal to the first posture of the first component at the first time.

[0021] In one embodiment, the transpose function is used to transfer the second pose of the second component to a first pose of the first component, wherein the first pose of the first component at a first time is equal to the transpose function multiplied by the second pose of the second component at the first time.

[0022] In one embodiment, the transpose function is solved by the head-mounted display device using the first posture, the second posture, and the displacement at each of a plurality of time points.

[0023] In summary, the embodiments of the present invention provide a virtual reality tracker and a tracker position correction method suitable for virtual reality systems. By dividing the wearable tracker into multiple components, the wearable tracker can be appropriately fixed to the user's body parts, such as fitting snugly on the user's wrist, enabling the processor of the head-mounted display device to accurately obtain the position of the tracker in three-dimensional space. Furthermore, the first and second cameras of the head-mounted display device are based on pre-calibrated camera settings, ensuring consistent integration of the coordinates between the cameras. Therefore, the processor of the head-mounted display device can calculate the positional correlation between the postures of multiple components by calculating the postures of each component. This positional correlation allows the coordinate systems of multiple components to be transformed to the coordinate system of a single component, achieving more accurate tracking of the posture and spatial position of each component. Attached Figure Description

[0024] Figure 1 This is a schematic diagram illustrating a virtual reality system according to an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram illustrating a tracker position correction method according to an embodiment of the present invention.

[0026] Figure 3 The flowchart illustrates a tracker position correction method according to an embodiment of the present invention.

[0027] Symbol explanation: 100: Virtual Reality System CAM1, CAM2: Cameras 11, 12, 21, 22: Light Emitting Diodes PA, PB: Components IMU: Light Emitting Diode TR: Virtual Reality Tracker T0: Initial Time T1: First Time dT: Attitude change 300: Head-mounted display device 310~340: Steps Detailed Implementation

[0028] The following description is a preferred embodiment of the invention and is intended to describe the basic spirit of the invention, but is not intended to limit the invention. The actual scope of the invention must be understood by referring to the claims that follow.

[0029] It must be understood that the terms "comprising" and "including" used in this specification are used to indicate the presence of specific technical features, values, method steps, work processes, elements and / or components, but do not preclude the addition of more technical features, values, method steps, work processes, elements, components, or any combination thereof.

[0030] The use of terms such as "first," "second," and "third" in the claims is to modify elements in the claims and is not intended to indicate a priority order, a prior relationship, or that one element precedes another, or the chronological order of the execution of method steps. It is only used to distinguish elements with the same name.

[0031] Please refer to Figure 1 , Figure 1 This is a schematic diagram illustrating a virtual reality system 100 according to an embodiment of the present invention. In one embodiment, the virtual reality system 100 includes a virtual reality tracker TR. In one embodiment, a plurality of cameras CAM1, CAM2 are included in a head-mounted display (HMD) (not shown). The head-mounted display further includes a storage device (not shown) and a processor (not shown).

[0032] In one embodiment, the storage device in the head-mounted display device may be a read-only memory, flash memory, floppy disk, hard disk, optical disk, USB flash drive, magnetic tape, a database accessible via a network, or a storage medium with the same function that can be easily conceived by those skilled in the art.

[0033] In one embodiment, the head-mounted display device includes a processor and multiple cameras (e.g., cameras CAM1, CAM2).

[0034] In one embodiment, the processor in the head-mounted display device may be implemented by a microcontroller, microprocessor, digital signal processor, application-specific integrated circuit (ASIC), or a logic circuit.

[0035] In one embodiment, the multiple cameras in the head-mounted display device include a first camera CAM1 and a second camera CAM2.

[0036] In one embodiment, multiple cameras in the head-mounted display device (e.g., cameras CAM1, CAM2) are positioned at different locations on the head-mounted display device to capture environmental images.

[0037] For ease of explanation, the multiple cameras in the head-mounted display device will be referred to below as... Figure 1 The cameras CAM1 and CAM2 in the example are used as examples.

[0038] In one embodiment, the first camera CAM1 and the second camera CAM2 can each be implemented by a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS).

[0039] In one embodiment, the virtual reality tracker TR includes part PA and part PB.

[0040] In one embodiment, the virtual reality tracker TR may include more components. For ease of explanation, the various components of the virtual reality tracker TR will be referred to below as... Figure 1 The components PA and PB in the present invention are used as examples; however, the number of components in the present invention is not limited thereto.

[0041] In one embodiment, component PA and component PB can each be implemented by a carrier. The carrier is made of materials such as plastic or carbon fiber, and the outer shell can be made of metal.

[0042] In one embodiment, component PA includes a plurality of first light-emitting diodes (LEDs) 11, 12. In one embodiment, component PA may include more first light-emitting diodes. For ease of explanation, the first light-emitting diodes of component PA will be referred to below as... Figure 1 The first light-emitting diode 11 and the first light-emitting diode 12 are used as examples.

[0043] In one embodiment, component PA includes an inertial measurement unit (IMU) for measuring an acceleration and a triaxial angular velocity of component PA.

[0044] In one embodiment, component PB includes a plurality of second light-emitting diodes (LEDs) 21, 22. In one embodiment, component PB may include more second light-emitting diodes. For ease of explanation, the second light-emitting diodes of component PB will be referred to below as... Figure 1 The second light-emitting diode 21 and the second light-emitting diode 22 are used as examples.

[0045] In one embodiment, when camera CAM1 captures a picture of component PA, multiple first light-emitting diodes 11 and 12 on component PA are in an illuminated state; when camera CAM2 captures a picture of component PB, multiple second light-emitting diodes 21 and 22 on component PB are in an illuminated state.

[0046] In this way, camera CAM1 can transmit multiple first images of component PA to the processor of the head-mounted display device for subsequent processing. The processor calculates the relative position of component PA to camera CAM1, or the position of component PA in the three-dimensional coordinate system of camera CAM1, based on the positions of the first light-emitting diodes 11 and 12 with brightness in the multiple first images. Similarly, camera CAM2 can transmit multiple second images of component PB to the processor of the head-mounted display device for subsequent processing. The processor calculates the relative position of component PB to camera CAM2, or the position of component PB in the three-dimensional coordinate system of camera CAM2, based on the positions of the second light-emitting diodes 21 and 22 with brightness in the multiple second images.

[0047] Since calculating the three-dimensional coordinates of an object in space based on multiple images is a known technique, such as three-dimensional object tracking algorithms, three-dimensional modeling algorithms, multi-view image synthesis algorithms, etc., it will not be elaborated here.

[0048] In one embodiment, component PA and component PB are connected by a flexible member 30. The flexible member 30 (indicated by slashes) can be, for example, a strap, elastic cord, fixing strap, belt, etc. Any accessory that can adjust the tightness so that component PA and component PB conform to the object can be used to realize the flexible member 30.

[0049] In one embodiment, the tracker TR uses a flexible component 30 (e.g., a strap) to fix components PA and PB to an object, such as a human body part like a wrist, arm, ankle, thigh, waist, etc. In the first embodiment, the wrist is taken as an example.

[0050] In one embodiment, the tracker TR uses a flexible component 30 (e.g., a strap) to fix components PA and PB to an object, such as a racket, bat, toy gun, steering wheel, etc.

[0051] In one embodiment, the inertial measurement unit (IMU) includes an accelerometer and a gyroscope. The accelerometer is used to measure acceleration. The gyroscope is used to measure triaxial angular velocity.

[0052] In one embodiment, the acceleration and triaxial angular velocity measured by the inertial measurement unit (IMU) are transmitted to an inertial computing program and a detailed attitude is calculated using an image backtesting integration method. The processor of the head-mounted display device generates a real-time image based on the detailed attitude.

[0053] In one embodiment, the inertial measurement unit (IMU) transmits the acceleration and triaxial angular velocity to an inertial calculation program via a transmission interface. The inertial calculation program calculates the detailed attitude based on the acceleration and triaxial angular velocity using an image attitude backtesting method. The head-mounted display device then generates a real-time image based on the detailed attitude.

[0054] In one embodiment, an accelerometer of an inertial measurement unit (IMU) is used to measure acceleration; and a gyroscope of the IMU is used to measure the three-axis angular velocity. The inertial calculation program calculates a detailed attitude based on the acceleration and the three-axis angular velocity using an image interpolation method, and transmits the detailed attitude to a head-mounted display (HMD) through a transmission interface. The HMD generates a real-time image based on the detailed attitude.

[0055] In one embodiment, when camera CAM1 and / or camera CAM2 fails to capture the first light-emitting diode 11, 12 or the second light-emitting diode 21, 22 that is not emitting light, the subsequent detailed attitude can be calculated using the integration method of an inertial computing program until the light-emitting object is observed. This allows for continuity between multiple first images and / or multiple second images.

[0056] In one embodiment, the inertial measurement unit (IMU) transmits acceleration and triaxial angular velocity to an inertial calculation program via a transmission interface. The camera CAM1 on the head-mounted display captures images of the first light-emitting diodes (LEDs) 11 and 12 on the PA component to generate multiple first images. The head-mounted display uses these first images, acceleration, and triaxial angular velocity to calculate a first pose of the first LEDs 11 and 12 in a three-dimensional space.

[0057] In one embodiment, the pose described in this invention includes a rotation matrix R and a translation vector T, wherein the rotation matrix R is synonymous with a three-dimensional rotation matrix: the rotation angle is given by three directional axes; the translation vector T is exactly the coordinate of the origin of the world coordinate system in the camera coordinate system, and in particular, it is generally represented by the symbol Tz to represent the "depth" of the origin of the world coordinate system in the camera coordinate system.

[0058] In one embodiment, the processor of the head-mounted display device can apply known algorithms, such as the 3D pose estimation-scaled orthogonal projection iterative transformation algorithm (POSIT), to calculate a first pose of the first light-emitting diodes 11, 12 in a three-dimensional space. However, the invention is not limited to this; any algorithm that can be applied to virtual reality space to calculate the pose of an object can be used.

[0059] In one embodiment, the detailed attitude can be the acceleration and triaxial angular velocity of component PA over a short period of time (approximately 1 to 2 microseconds).

[0060] In one embodiment, the head-mounted display device transmits a first attitude to an inertial calculation program in the program. The inertial measurement unit (IMU) transmits acceleration and triaxial angular velocity to the inertial calculation program through a transmission interface. Based on the first attitude, acceleration, and triaxial angular velocity, the inertial calculation program calculates the detailed attitude using the image attitude backtesting integration method. The head-mounted display device generates a real-time image based on the detailed attitude.

[0061] Therefore, assuming that the camera CAM1 has a frame rate of 30 frames per second, it is insufficient and cannot provide the head-mounted display with the content. Real-time images generated based on detailed postures can help to provide the head-mounted display with the content in real time.

[0062] In one embodiment, camera CAM1 or camera CAM2 on the head-mounted display device captures images of the second light-emitting diodes 21 and 22 on the component PB to generate multiple second images. The head-mounted display device uses these second images to calculate a second pose of the second light-emitting diodes 21 and 22 in three-dimensional space.

[0063] In one embodiment, in virtual reality, components PA and PB can be considered to exist in the same three-dimensional space. However, a first coordinate system for the first pose corresponds to camera CAM1, and a second coordinate system for the second pose corresponds to camera CAM2; the first and second coordinate systems are different. The relative transposes of cameras CAM1 and CAM2 during the construction of the head-mounted display device are established, thus allowing the relative positional relationship between components PA and PB to be determined.

[0064] In one embodiment, cameras CAM1 and CAM2 are located on the same head-mounted display device. The distance transformation between cameras CAM1 and CAM2 is defined when the head-mounted display device is constructed. Therefore, the transformation between the first coordinate system and the second coordinate system is a known universal coordinate system. All attitude calculations are based on the universal coordinate system.

[0065] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating a tracker position correction method 200 according to an embodiment of the present invention. When the wrist wearing the tracker TR moves from position T0 to position T1, cameras CAM1 and CAM2 of the head-mounted display device capture images of components PA and PB at T0 and T1. The head-mounted display device then calculates the first pose of component PA and the second pose of component PB at T0 and T1, as well as the positional correlation between components PA and PB.

[0066] In one embodiment, please refer to Figure 2 The soft component 30 is a strap. When component PA and component PB are strapped to an object (e.g., a wrist) by the strap 30, the first posture of component PA at the initial time T0 multiplied by a displacement is considered to be equal to the first posture of component PA at the first time T1. And the second posture of component PB at the initial time T0 multiplied by the displacement and then multiplied by a transpose function, the result obtained will be equal to the first posture of the first component at the first time T1.

[0067] In one embodiment, the flexible component 30 is a strap. After components PA and PB are strapped to an object, the tracker position correction method further includes: multiplying a first posture of component PA at an initial time T0 by a rotational displacement, which is considered equal to a first posture of component PB at a first time T1; and multiplying a second posture of component PB at an initial time T0 by a rotational displacement, which is considered equal to a second posture of component PB at a first time T1. The second posture is then multiplied by a transpose function, and the calculated result is equal to the first posture of component PA at a first time T1. In one embodiment, the transpose function is used to transfer the second posture of component PB to a first posture of component PA, and the first posture of component PA at a first time T1 is equal to the transpose function multiplied by the second posture of component PB at a first time T1.

[0068] In one embodiment, please refer to Figure 2 The flexible component 30 is a strap. After components PA and PB are strapped to an object (e.g., a wrist) via the strap 30, the processor of the head-mounted display performs the following calculations: Between an initial time T0 and a first time T1, the inertial calculation program continuously calculates the detailed attitude of component PA. The first attitude calculated by component PA at the first time has the detailed attitude content referenced by the inertial calculation program, thus the first attitude has high accuracy. Since the tracker TR, component PA, and component PB are a single unit, the attitude change dT (a rotation matrix and a displacement vector) of component PA from the initial time T0 to the first time T1 can be considered as the attitude change of the tracker TR itself.

[0069] More specifically, cameras CAM1 and CAM2 each capture images of components PA and PB at times T0 and T1, respectively, for example, at a frame rate of 30. The processor of the head-mounted display calculates the first pose of component PA and the second pose of component PB at times T0 and T1, respectively. Assuming that the displacement dT of component PA from time T0 to time T1 is close to its transpose function, we can obtain: dT * Pose(PA:T0) ~= Pose(PA:T1), where Pose(PA:T0) represents the first pose of component PA at time T0, and Pose(PA:T1) represents the first pose of component PA at time T1.

[0070] Since the first coordinate system of the first posture corresponds to camera CAM1, and the second coordinate system of the second posture corresponds to camera CAM2, and since components PA and PB can only be considered as a single rigid body after being fixed by strap 30, the relative positional relationship of component PB transposed to component PA can only be measured at this time. This transpose function will be represented by the symbol TBA below.

[0071] Generally, when components PA and PB are both captured by camera CAM1 in the same image, the transpose function TBA can be easily calculated using the formula: [pose B] - 1 [pose A], where "pose B" represents the second pose of component PB and "pose A" represents the first pose of component PA. However, component PB lacks an inertial measurement unit (IMU), and the second pose can only be calculated using the second LEDs 21 and 22 in the image. The error in the second pose is relatively large, potentially leading to inaccurate TBA calculated using this formula. In other words, because components PA and PB are connected, component PA contains an IMU, allowing for the calculation of the first pose with high accuracy. Component PB, lacking an IMU, has a certain degree of error in its second pose. However, since components PA and PB are considered a single rigid body after being fixed by strap 30, the attitude change dT calculated for component PA can be considered as the attitude change of component PB. If the second attitude error calculated by component PB at the initial time T0 and the first time T1 is sufficiently small, the attitude change of the second attitude will be close to dT. Therefore, dT * TBA * Pose(PB:T0) ~= TBA * Pose(PB:T1) will be satisfied. The symbol Pose(PB:T0) represents the second attitude of component PB at time T0, and the symbol Pose(PB:T1) represents the second attitude of component PB at time T1.

[0072] The head-mounted display device can obtain the following by calculating the displacement dT1~dTn over multiple time intervals T1~Tn: dT1 * Pose(PA:T0) ~= Pose(PA:T1) dT2 * Pose(PA:T1) ~= Pose(PA:T2) … dTn * Pose(PA:Tn-1) ~= Pose(PA:Tn) Wherein, dT2 represents the displacement of tracker TR at times T1 to T2, dTn represents the displacement of tracker TR at times Tn-1 to Tn, Pose(PA:T2) represents the first pose of component PA at time T2, Pose(PA:Tn-1) represents the first pose of component PA at time Tn-1, and Pose(PA:Tn) represents the first pose of component PA at time T.

[0073] Since TBA is a transpose function of component A and component B, and by combining the above calculation formulas, it can be deduced that dTn * TBA * Pose(PB:Tn) ~=TBA * Pose(PB:Tn+1), where the symbol Pose(PB:Tn) represents the second pose of component PB at time Tn, and the symbol Pose(PB:Tn+1) represents the second pose of component PB at time Tn+1.

[0074] The movement positions of components PA and PB in every two frames yield a transpose function TBA. By integrating multiple transpose functions, the transpose function TBA can be defined more precisely. Based on the user's actions and the images captured at each time point, the following correction function can be determined: T(BnAn) = Pose(Btn) -1 *Pose(Atn) When the correction function begins to be processed by the processor of the head-mounted display, the user is asked to move his wrist for a period of time to obtain multiple samples of the transpose function TBA. The final transpose function TBA is then calculated using a weighted average of these samples.

[0075] For each transpose function TBA, the weight Wn can be set as follows:

[0076] Here, the symbol T(AnAn+1) represents the attitude change of component A from time N to time N+1, the symbol T(BnBn+1) represents the attitude change of component B from time N to time N+1, the symbol Pose(Atn) represents the first attitude of component A at time N, the symbol Pose(Atn+1) represents the first attitude of component A at time N+1, the symbol Pose(Btn) represents the second attitude of component B at time N, and the symbol Pose(Btn+1) represents the second attitude of component B at time N. By using the weights Wn of the samples of multiple transpose functions TBA, the average of these weights Wn can be calculated to obtain the final transpose function TBA.

[0077] In one embodiment, to ensure that the weights Wn of each transpose function TBA sample are good enough (where n represents the nth transpose function TBA), the processor only collects weights between 0.95 and 1 and calculates the average.

[0078] The final transpose function TBA is calculated by multiplying the TBA of each group by its corresponding weight and then dividing by the sum of the total weights. The definition is as follows:

[0079] In this context, the symbol T(BA) represents the formula for the final transpose function TBA, the symbol Sum(w) represents the sum of all weights, the symbol normal(Wn) represents the weight of the nth group divided by the sum of weights, the symbol T(BnAn) represents the TBA of the nth group, and sum is the summation function; that is, summing from n=1 to all weights. Therefore, the symbol T(BA) is regarded as the final transpose function TBA, and the following calculations will use this transpose function TBA.

[0080] In one embodiment, the head-mounted display device solves for the transpose function TBA by taking the first posture, the second posture, and the displacement dT at each of the plurality of time points.

[0081] In one embodiment, after the head-mounted display device completes the calibration process, i.e., the TBA calculation, component PB can use the TBA to calculate the coordinate system position of each LED from component PA to each second light-emitting diode (i.e., component PB).

[0082] Please see Figure 3 , Figure 3 The flowchart illustrates a tracker position correction method 300 according to an embodiment of the present invention. The tracker position correction method 300 is applicable to virtual reality systems.

[0083] In step 310, a first camera (such as camera CAM1) is used to capture images of a plurality of first light-emitting diodes (such as first light-emitting diodes 11 and 12) on a first component (such as component PA) to obtain a plurality of first images.

[0084] In step 320, an acceleration and a triaxial angular velocity of the first component are measured by an inertial measurement unit (such as an inertial measurement unit IMU).

[0085] In step 330, a second camera (such as camera CAM2) is used to capture images of a plurality of second light-emitting diodes (such as second light-emitting diodes 21 and 22) of a second component (such as component PB) to obtain a plurality of second images.

[0086] In step 340, the processor of the head-mounted display device uses the first images, acceleration, triaxial angular velocity, and the second images to correct the position of each light-emitting diode (LED, such as the first LED and the second LED) on a tracker. The first and second components are connected by a flexible component (such as flexible component 30).

[0087] In one embodiment, the first camera and the second camera can be the same camera, as long as the camera's shooting angle is wide enough to cover the range of multiple components on the tracker TR. In other words, the number of cameras is not limited; the number of cameras depends on the range of multiple components on the tracker TR that can be captured.

[0088] In summary, the embodiments of the present invention provide a virtual reality tracker and a tracker position correction method suitable for virtual reality systems. By dividing the wearable tracker into multiple components, the wearable tracker can be appropriately fixed to the user's body parts, such as fitting snugly on the user's wrist, enabling the processor of the head-mounted display device to accurately obtain the position of the tracker in three-dimensional space. Furthermore, the first and second cameras of the head-mounted display device are based on pre-calibrated camera settings, ensuring consistent integration of the coordinates between the cameras. Therefore, the processor of the head-mounted display device can calculate the positional correlation between the postures of multiple components by calculating the postures of each component. This positional correlation allows the coordinate systems of multiple components to be transformed to the coordinate system of a single component, achieving more accurate tracking of the posture and spatial position of each component.

[0089] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the scope of the invention. Any person skilled in the art may make some modifications without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A virtual reality tracker, comprising: A first component, the first component comprising: Multiple first light-emitting diodes; as well as An inertial measurement unit is used to measure the acceleration and triaxial angular velocity of the first component; and A second component, the second component comprising: Multiple second light-emitting diodes; The first component and the second component are connected by a flexible member; The inertial measurement unit transmits the acceleration and the triaxial angular velocity to a head-mounted display device. The head-mounted display device captures images of the plurality of first light-emitting diodes on the first component to generate a plurality of first images; The head-mounted display device calculates a first posture of the multiple first light-emitting diodes in a three-dimensional space using the multiple first images, the acceleration, and the three-axis angular velocity. When the first component and the second component are attached to an object, the first orientation of the first component at an initial time multiplied by a displacement is considered equal to the first orientation of the first component at the first time; and Multiplying the second component's second attitude at the initial time by the displacement, and then multiplying it by a transpose function, the resulting calculation will be equal to the first component's first attitude at the first time.

2. The virtual reality tracker of claim 1, wherein, The inertial measurement unit includes: An accelerometer, used to measure the acceleration; and A gyroscope is used to measure the angular velocity of the three axes; The inertial measurement unit transmits the acceleration and the three-axis angular velocity to an inertial calculation program through a transmission interface. The inertial calculation program calculates a detailed attitude using an image attitude backtesting method based on the acceleration and the three-axis angular velocity. A head-mounted display device generates a real-time image based on the detailed attitude.

3. The virtual reality tracker of claim 2, wherein the inertial measurement unit transmits the acceleration and the triaxial angular velocity to a head-mounted display device via a transmission interface; A first camera on the head-mounted display captures images of the plurality of first light-emitting diodes on the first component to generate a plurality of first images.

4. The virtual reality tracker as claimed in claim 3, wherein the head-mounted display device transmits the first posture to the inertial calculation program among multiple programs, the inertial measurement unit transmits the acceleration and the three-axis angular velocity to the inertial calculation program through the transmission interface; the inertial calculation program calculates a detailed posture based on the first posture, the acceleration and the three-axis angular velocity using an image posture backtesting integration method, and the head-mounted display device generates a real-time image based on the detailed posture.

5. The virtual reality tracker of claim 3, wherein the first camera or a second camera on the head-mounted display captures images of the plurality of second light-emitting diodes on the second component to generate a plurality of second images; and The head-mounted display device uses the multiple second images to calculate a second orientation of the multiple second light-emitting diodes in the three-dimensional space.

6. The virtual reality tracker of claim 5, wherein a first coordinate system of the first pose corresponds to the first camera, and a second coordinate system of the second pose corresponds to the second camera, wherein the first coordinate system and the second coordinate system are different.

7. The virtual reality tracker as claimed in claim 6, wherein the first camera and the second camera are located in the same head-mounted display device, and the distance conversion between the first camera and the second camera is defined when the head-mounted display device is constructed, so the first coordinate system and the second coordinate system can be converted into a known universal coordinate system, and all calculations of posture are based on the universal coordinate system.

8. The virtual reality tracker of claim 3, wherein the soft component is a strap, and the first component and the second component are attached to the object by the strap.

9. A tracker position correction method, applicable to a virtual reality system, wherein the tracker position correction method includes: To capture multiple first light-emitting diodes on a first component, thereby obtaining multiple first images; An inertial measurement unit is used to measure the acceleration and triaxial angular velocity of the first component; To capture multiple second light-emitting diodes of a second component to obtain multiple second images; and The position of a tracker is corrected based on the multiple first images, the acceleration, the three-axis angular velocity, and the multiple second images; The first component and the second component are connected by a flexible member; The tracker includes the virtual reality tracker as described in any one of claims 1-8.

10. The tracker position correction method as described in claim 9, further comprising: The acceleration is measured using an accelerometer within the inertial measurement unit; and The angular velocity of the three axes is measured using a gyroscope in the inertial measurement unit. in, The inertial measurement unit calculates a detailed attitude based on the acceleration and the three-axis angular velocity using an image interpolation method, and transmits the detailed attitude to a head-mounted display device through a transmission interface. The head-mounted display device generates a real-time image based on the detailed attitude.

Images