OBJECT TRACKING SYSTEM AND OBJECT TRACKING METHOD

Abstract

Object tracking system comprising: a head-mounted display (HMD) comprising: a first processor configured to transmit search information for searching for an object during a search phase; an accessory device connected to the HMD device, comprising: at least one accessory camera; and a second processor connected to the at least one accessory camera and configured to receive the search information and determine whether one of the at least one accessory cameras captured an object image during the search phase; wherein, when the second processor determines that at least one of the at least one accessory camera has captured the object image, the second processor transmits notification information to the first processor and the first processor enters a tracking phase and transmits request information to the second processor; when the second processor receives the request information, the second processor performs one of the following actions: Transmitting the object image to the first processor, wherein the first processor calculates an object pose according to the object image; and Calculating the object position based on the object image and transmitting the object position to the first processor.

Description

BACKGROUND OF THE INVENTIONField of the invention

[0001] The present disclosure relates to a tracking system and, more particularly, to an object tracking system and an object tracking method. Description of the state of the art

[0002] Existing camera-based head-mounted displays (HMDs) have a fixed number of cameras. For example, two cameras can be located in the front center, one camera on the right side, and another on the left side of the camera-based head-mounted display. This fixed number of cameras can capture images within a specific field of view, e.g., a horizontal field of view.

[0003] US 2016 / 0063762 A1 describes methods for managing content in an interactive augmented reality environment. An augmented reality environment may be provided to an end user of a head-mounted display (HMD) device, in which content (e.g., web pages) may be displayed to the end user via one or more curved writing tablets positioned on a virtual cylinder that appears body-bound to the end user. The virtual cylinder may be arranged around the end user, with the end user positioned at the center of the virtual cylinder, such that the one or more curved writing tablets appear to be displayed at the same distance from the end user. The position and size of each of the one or more curved writing tablets may be controlled by the end user using head gestures and a virtual pointer projected onto the virtual cylinder.

[0004] However, a fixed number of cameras can only provide a limited field of view, e.g., a vertical field of view. This can limit the controller tracking range, making it impossible to capture areas above and below the camera-based head-mounted display.

[0005] Therefore, the question of how to add at least one camera module to an existing module, such as a camera-based head-mounted display, in order to expand the field of view of the camera-based head-mounted display has become one of the problems that need to be solved in this field. BRIEF SUMMARY OF THE INVENTION

[0006] In accordance with one feature of the present invention, the present disclosure provides an object tracking system. The object tracking system includes a head-mounted display (HMD) and an accessory device. The HMD device includes a first processor. The first processor is configured to communicate search information for searching for an object during a search phase. The accessory device is connected to the HMD device. The accessory device includes at least one accessory camera and a second processor. The second processor is connected to the at least one accessory camera. The second processor is configured to receive the search information and determine whether one of the at least one accessory cameras captured an object image during the search phase.When the second processor determines that at least one of the at least one accessory camera has captured the object image, the second processor sends notification information to the first processor, and the first processor enters a tracking phase and sends request information to the second processor. When the second processor receives the request information, the second processor performs one of the following actions: transmitting the object image to the first processor, wherein the first processor calculates an object pose according to the object image; and calculating the object pose according to the object image and transmitting the object pose to the first processor.

[0007] In accordance with one feature of the present invention, the present disclosure provides a method for tracking objects. The object tracking method includes the steps of: transmitting, by a first processor, search information for searching for an object during a search phase; receiving the search information and using a second processor to determine whether one of at least one accessory camera captured an object image during the search phase. If the second processor determines that at least one of the at least one accessory camera captured the object image, the second processor sends notification information to the first processor, and the first processor enters a track phase and sends request information to the second processor.When the second processor receives the request information, the second processor performs one of the following actions: transmitting the object image to the first processor, wherein the first processor calculates an object pose according to the object image; and calculating the object pose according to the object image and transmitting the object pose to the first processor.

[0008] Embodiments of the present invention provide an object tracking system and an object tracking method equipped with modularized accessory cameras of an accessory device. The accessory cameras are pluggable as needed. An additional processor in the accessory device could help the head-mounted display device search for and track the controller. In this way, the object tracking system and the object tracking methods achieve the effect of better distributed computing power. Furthermore, the object tracking system and the object tracking methods can flexibly add at least one camera module (i.e., an accessory camera) to an existing module (i.e., a head-mounted display device) to expand the field of view of the head-mounted display device. The object tracking range (e.g., controller tracking range) can also be expanded. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The invention can be better understood from the following detailed description and examples with reference to the accompanying drawings, in which the Fig. 1-2 are schematic representations of an object tracking system in accordance with an embodiment of the present disclosure; Fig. 3 is a schematic of a front view of an accessory mounted on the object tracking system according to an embodiment of the present disclosure; Fig. 4A is a block diagram of the object tracking system in accordance with an embodiment of the present disclosure; Fig. 4B is a block diagram of a controller in accordance with an embodiment of the present disclosure; Fig. 5 is a flowchart of a method for powering on the accessory device according to an embodiment of the present disclosure; Fig. 6 is a flowchart of an object tracking method in accordance with an embodiment of the present disclosure; Fig. 7 is a schematic of object tracking in accordance with an embodiment of the present disclosure; and Fig. 8 is a flowchart of an object tracking method in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following description describes the best mode contemplated for carrying out the invention. This description is intended to illustrate the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

[0011] The present invention will be described with respect to specific embodiments and with reference to specific drawings, but the invention is not limited thereto and is limited only by the claims. It is further understood that the terms "comprises," "includes," and / or "include," when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0012] The use of ordering terms such as "first", "second", "third", etc. in the claims to modify a claim element does not, in itself, establish any priority, precedence, or ordering of one claim element over another, or the chronological order in which the acts of a method are performed, but is used merely as a label to distinguish one claim element with a particular name from another element with the same name (but using the ordering term) to distinguish the claim elements.

[0013] With reference to Fig. 1-4A and 4B are Fig. 1-2 are schematic representations of an object tracking system 100 in accordance with an embodiment of the present disclosure. Fig. 3 is a schematic representation of a front view of an attachment mounted on the object tracking system 100 according to an embodiment of the present disclosure. Fig. 4A is a block diagram of the object tracking system 100 in accordance with one embodiment of the present disclosure. Fig. 4B is a block diagram of a control device CR in accordance with an embodiment of the present disclosure.

[0014] In one embodiment, a user USR wears the head-mounted display device HMD to experience virtual reality. In one implementation, the head-mounted display device HMD can be attached to the user's head via a headband HB or a helmet.

[0015] In one embodiment, the object tracking system 100 includes a head-mounted display (HMD) and an accessory device (COV). The accessory device (COV) connects to the head-mounted display (HMD). In one embodiment, the accessory device (COV) can be any wearable device, such as a smart watch, a camera module, a smart wristband, a sensor module, a masking device, etc. The accessory device (COV) includes at least one accessory camera and a processor.

[0016] In one version, the accessory device COV is a front cover device (as in Fig. 1) for the head-mounted display device HMD, the main cameras C0-C3 include a left side camera C2, a right side camera C3, a left front camera C0, and a right front camera C1. The at least one accessory camera C4-C5 includes a camera mounted on the top of the front cover device and another mounted on the bottom of the front cover device.

[0017] With reference to Fig. 1 and Fig. 4A, the head-mounted display device HMD in one embodiment includes a processor P1 and at least one main camera C0-C3. Each main camera C0-C3 is connected to the processor P1 (as shown in Fig. 4A). In one embodiment, the main cameras C0-C3 are a left side camera C2, a right side camera C3, a left front camera C0, and a right front camera C1 on the head-mounted display (HMD). For ease of explanation, four main cameras C0-C3 are used in the example illustrated in the following embodiments. However, those skilled in the art will understand that the present invention is not limited to the initial configuration of four cameras in the head-mounted display (HMD).

[0018] In one embodiment, the accessory device COV comprises a processor P2 and at least one accessory camera C4-C5 (i.e., the accessory camera). Each accessory camera C4-C5 is connected to the processor P2 (as shown in Fig. 4A). In one embodiment, the accessory device COV may be a cover with embedded accessory cameras C4-C5 that connect to the processor P2. In one embodiment, one of the accessory cameras (e.g., the accessory camera C4) is mounted on the top of the accessory device COV, and another accessory camera (e.g., the accessory camera C5) is mounted on the bottom of the accessory device COV. For clarity, the examples used in the following embodiments are equipped with two accessory cameras C4-C5. However, those skilled in the art will understand that the present invention is not limited to configuring two accessory cameras in the accessory device COV.

[0019] In one embodiment, the accessory cameras C4-C5 can be camera modules that are mounted / installed in the accessory device COV.

[0020] In one version, the COV accessory device contains holes H1 and H0. The position of hole H1 corresponds to the position of the main camera C1. The position of hole H0 corresponds to the position of the main camera C0.

[0021] In one embodiment, processor P1 and / or processor P2 may be any electronic device that has a computing function. Processor P1 and / or processor P2 may be implemented by an integrated circuit, such as a microcontroller, a microprocessor, a digital signal processor, an application-specific integrated circuit (ASIC), or a logic circuit.

[0022] In one embodiment, the main cameras C0-C3 and the accessory cameras C4-C5 may be implemented by a camera module containing a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS).

[0023] In one embodiment, the accessory COV is connected to the head-mounted display (HMD) via a wire or a wireless connection. The accessory COV can, for example, be connected to the head-mounted display (HMD) using the serial peripheral interface (SPI). In this way, information can be exchanged between the processors P1 and P2 that connect the accessory COV to the head-mounted display (HMD) (as shown in Fig. 2).

[0024] In a version as in Fig. 3, when the accessory device COV is connected to the head-mounted display device HDM, the accessory camera C4 is positioned higher than the left front camera C0 and the right front camera C1, and the accessory camera C4 is positioned lower than the left front camera C0 and the right front camera C1.

[0025] In one embodiment, the accessory cameras C4-C5 are selectively placed at different locations to expand the field of view (FOV) of the head-mounted display (HMD). The processor P1 can receive an object image (e.g., a controller image) from an accessory camera (e.g., accessory camera C5) or a main camera (e.g., main camera C0).

[0026] In other words, the accessory device COV can be considered as a cover device with the upper and lower accessory cameras C4-C5. As shown in Fig. As shown in Figure 2, the accessory device COV can be connected to the head-mounted display HMD. The pluggable accessory cameras C4-C5 can expand the vertical field of view (e.g., up to 220 degrees). Therefore, the object tracking range can be expanded by adding the accessory cameras C4-C5. For clarity, the following explanations use the controller CR as an example of the object. However, those skilled in the art should understand that the present invention is not limited to tracking the controller CR.

[0027] In one embodiment, as shown in Figure 4B, the controller CR includes light-emitting diodes (LEDs) 40. The processor P1 or the processor P2 calculates the object position (e.g., the controller position) based on the relative position of the LEDs 40 in the object image (e.g., the controller image). In one embodiment, these cameras C0-C5 can capture these LEDs 40 frame by frame.

[0028] In one embodiment, the object image is an image of the object. And the object image may contain only the partial image with identifiable features of the object. The identifiable feature may be, for example, a light sphere, the LEDs 40, or another specific pattern.

[0029] In one embodiment, the LEDs 40 are arranged in a pattern (see Fig. 7), and the processor P1 or P2 can determine the distance between the head-mounted display HMD and the controller CR by the distance of each line segment of the pattern.

[0030] In one embodiment, the LEDs 40 are arranged in a pattern, and the processor P1 or P2 can determine the controller position based on the rotation angle of the pattern. The processor P1 or P2 can apply known algorithms to calculate the object pose (e.g., the controller pose).

[0031] In one embodiment, the object pose is the pose of the object. The object pose includes the rotation information and the coordinates of the object. For example, the processor P1 or P2 can define the rotation and the absolute coordinates of the X, Y, and Z axes of the controller CR in three-dimensional real space according to the pattern in the controller image (e.g., the position of the line segment in the pattern) and the position of the head-mounted display device HMD. Then, the processor P1 or P2 can track the movement of the controller CR on the absolute coordinates of the X, Y, and Z axes and the rotation of the controller CR, and thereby calculate the relative coordinate between the controller CR and the head-mounted display device HMD in virtual reality.However, those of ordinary skill should understand that the present invention is not limited to tracking the controller CR with the LEDs 40, nor is it limited to using the above methods. Furthermore, the processor P1 or P2 may use various known tracking algorithms to track different objects.

[0032] With reference to Fig. 5-7 is Fig. 5 is a flowchart of a method 500 for switching on the auxiliary device according to an embodiment of the present disclosure. Fig. 6 is a flowchart of an object tracking method 600 in accordance with an embodiment of the present disclosure. Fig. 7 is a schematic of object tracking according to an embodiment of the present disclosure. Fig. 8 is a flowchart of an object tracking method 800 in accordance with an embodiment of the present disclosure.

[0033] In one embodiment, the object tracking method 600 and / or the object tracking method 800 may be applied to track multiple objects. For simplicity, the controller CR is used as an example below.

[0034] With reference to Fig. 5, the head-mounted display HMD detects whether the accessory COV is connected to the head-mounted display HMD in step 510. If the head-mounted display HMD detects that the accessory COV is connected to the head-mounted display HMD, step 520 is executed. If the head-mounted display HMD determines that the accessory COV is not connected to the head-mounted display HMD, step 510 is executed again. In step 520, the head-mounted display HMD supplies power to the accessory COV. Therefore, the accessory COV is turned on in step 520. Thus, the power supply for the accessory COV is provided by the head-mounted display HMD. Once the accessory COV is connected to the head-mounted display HMD, the accessory COV is in the turned-on state.

[0035] With reference to Fig. 6, the object tracking method 600 is performed by the processor P1 of the head-mounted display device HMD and the processor P2 of the auxiliary device COV.

[0036] In step 610, the processor P1 enters the search phase for searching for an object image (e.g., the controller CR).

[0037] In one embodiment, the processor P1 determines whether one of the at least one main camera C0-C3 has captured an object image during the search phase.

[0038] In step 620, the processor P1 transmits search information for searching for the object during a search phase.

[0039] In step 630, the processor P2 receives the search information.

[0040] In one embodiment, processor P2 receives the search information and executes a search algorithm. The search algorithm for the object image can use known search algorithms, such as the perceptual hash algorithm, feature matching algorithm, etc.

[0041] In one embodiment, processor P1 executes the search algorithm for the images captured by a main camera C0-C3, and processor P2 executes the search algorithm for the images captured by an accessory camera C4-C5 to determine whether the object image was captured during the search phase.

[0042] In one embodiment, the object image may be an image including the controller image.

[0043] In one embodiment, the processor P1 checks all images captured by the main cameras C0-C3 to find the object image.

[0044] In one embodiment, processor P2 examines all images captured by accessory cameras C4-C5 to find the object image. For example, processor P2 compares which part of each image is like a controller CR from the images captured by accessory cameras C4-C5 to find the controller image.

[0045] In step 640, processor P2 uses a second processor to determine whether one of at least one accessory camera C4-C5 has captured an object image.

[0046] In one embodiment, processor P2 determines whether at least one accessory camera C4-C5 connected to processor P2 captured an object image during the search phase. If processor P2 determines that at least one accessory camera C4-C5 connected to processor P2 captured the object image during the search phase, step 650 is executed.

[0047] In step 650, processor P2 transmits notification information to processor P1.

[0048] In one embodiment, the notification information also includes camera information. The camera information indicates that a specific accessory camera (e.g., accessory camera C5) captured the object image. Since processor P1 has received the camera information, step 660 is executed.

[0049] In step 660, processor P1 enters the tracking phase.

[0050] In one embodiment, processor P1 continuously tracks or calculates the object position of the object image captured by the specific accessory camera (e.g., accessory camera C5) during the tracking stage. Therefore, processor P1 must inform the accessory device COV to provide the object image.

[0051] In one embodiment, the processor P2 transmits handover information to the head-mounted display device HMD when the object moves from the FOV of one of the cameras C0-C5 (e.g., the FOV of the accessory camera C4) to another (e.g., the FOV of the accessory camera C5). The handover information indicates that another specific accessory camera (e.g., the accessory camera C4) has captured the object image. The processor P1 performs a handover process to continuously track or calculate the object position of the object image captured by the other specific accessory camera (e.g., the accessory camera C4) in the tracking phase. As shown in Fig. As shown in Figure 7, when the controller CR moves upward from the bottom of the head-mounted display HMD (the arrow indicates that the controller CR moves from bottom to top), and the controller image is first captured by the camera C5 and changes to the capture of camera C4, the processor P1 performs a handover process to continuously track or calculate the controller position of the controller image captured by the accessory camera C4 in the tracking phase.

[0052] In step 670, processor P1 transmits request information to processor P2. In one embodiment, processor P1 transmits request information to processor P2 to request the auxiliary device COV to provide the object image or multiple object images.

[0053] In step 680, when the processor P2 receives the request information, the processor P2 prepares the object image to be sent.

[0054] In one embodiment, the request information includes a cropping command. When processor P2 receives the request information, processor P2 crops the object image to a sub-image of the object image (e.g., retrieving / cropping the controller image portion from the entire object image) that still contains the object and transmits the sub-image (e.g., the controller image portion) to processor P1. Processor P1 calculates the object pose according to the sub-image. For example, the entire object image is 640*480 pixels. Processor P2 may crop the controller image portion (e.g., 60*40 pixels) from the entire object image. By cropping the object portion, it can reduce the transmission amount in step 690.

[0055] In one embodiment, the partial image may contain only the partially identifiable feature of the object. For example, the partial image may contain only a light sphere, some of the LEDs 40, or a partial specific pattern, without an entire controller CR image.

[0056] In other words, once the controller image is found, the search phase shifts to the tracking phase. The processor P1 of the head-mounted display (HMD) sends the crop command to the processor P2 to request the provision of the object images. Furthermore, the crop command means that the auxiliary device (COV) only sends the object image region containing the controller image (i.e., the controller image), not the entire object image. With this technology, image data transmission can be obtained immediately.

[0057] In step 690, processor P2 transmits the object image to processor P1.

[0058] In one embodiment, processor P2 transfers the object image to a buffer (e.g., RAM) of the head-mounted display device (HMD) as soon as an accessory camera (e.g., accessory camera C5) has captured the object image. Processor P1 can retrieve the object image from the buffer.

[0059] In step 695, the processor P1 tracks or calculates the object position from the object image.

[0060] In one embodiment, after receiving one or more object images, the processor P1 tracks or calculates the object pose based on the object image.

[0061] In one embodiment, if processor P1 does not continuously receive the object image, it reenters the search phase (step 610). For example, if processor P1 loses tracking of the object, it reenters the search phase. Then, all main cameras C0-C3 and / or all accessory cameras C4-C5 reenter the search phase for the object image.

[0062] In one embodiment, the search phase is used to use several or all cameras C0-C5 to determine which camera captured the object image. Furthermore, the search phase is used to determine in which part of the entire FOV the object image appears.

[0063] For example, after the controller CR first leaves the FOV and then the controller CR re-enters the FOV, the processor P1 enters the search phase to find out which specific camera the controller image was taken from.

[0064] In another example, during the search phase, if processor P1 knows that controller CR is moving out of the FOV from the bottom of the FOV, it predicts that controller CR will move back from the bottom of the FOV. Therefore, processor P1 increases the priority of the search weight of accessory camera C5. However, processor P1 continues to search all cameras C0-C5 for controller images.

[0065] In one embodiment, after the specific camera (e.g., accessory camera C5) has retrieved the object image in the search phase, the tracking phase is used to accurately track the object pose from the specific camera (e.g., accessory camera C5). During the tracking phase, other cameras (e.g., cameras C0-C4) continuously capture images, but the images captured by them are not stored in the cache and are not used for further calculations.

[0066] Therefore, by applying the camera extension concept, at least one COV can be mounted or connected to the head-mounted display (HMD). The connection methods of the COV and the head-mounted display (HMD) are not limited to this.

[0067] Fig. 8 is a flowchart of an object tracking method 800 in accordance with an embodiment of the present disclosure. Steps 610-660 are the same as steps 810-860, respectively. Therefore, steps 870-890 are described below.

[0068] The Fig. 6 and Fig. The object image mentioned in 8 is captured by at least one accessory camera C4-C5. The difference is that the object position is determined by the processor P1 in Fig. 6 and the object position from processor P2 in Fig.8 is calculated. In step 870, processor P1 transmits the request information to processor P2.

[0069] In one embodiment, the processor P1 sends request information to the processor P2 to request the auxiliary device COV to provide the object pose according to one or more object images.

[0070] For example, when the controller image is found, the search stage switches to the tracking stage, the processor P1 of the head-mounted display HMD sends the tracking command to processor P2 and requests the controller position.

[0071] In step 880, when the processor P2 receives the request information, the processor P2 calculates the object pose according to one or more object images.

[0072] If the processor P2 has better computing power, it can calculate the object position based on one or more object images.

[0073] In step 890, processor P2 transmits the object pose to processor P1.

[0074] In one embodiment, processor P2 transmits the object position to a buffer (e.g., RAM) of the head-mounted display device (HMD) as soon as an accessory camera (e.g., accessory camera C5) has captured the object image. Processor P1 can retrieve the object pose from the buffer.

[0075] For example, the processor P2 defines the rotation and absolute coordinates of the X, Y, and Z axes of the controller CR in three-dimensional real space according to the pattern in the controller image and the position of the head-mounted display HMD.

[0076] In step 895, the processor P1 uses the object pose obtained from the processor P2, or the processor P1 calculates the object pose according to the object image captured by the at least one main camera C0-C3.

[0077] For example, the processor P1 tracks the movement of the controller CR on the absolute coordinate of the X, Y and Z axes and the rotation of the controller CR and thus calculates the relative coordinate between the controller CR and the head-mounted display device HMD in virtual reality.

[0078] In one embodiment, when the at least one main camera C0-C3 captures the object image, the processor P1 calculates the object pose according to the object image captured by the at least one main camera C0-C3. When the at least one accessory camera C4-C5 captures the object image, the processor P1 or P2 calculates the object pose according to the object image captured by the at least one accessory camera C4-C5 (according to the object tracking method 600 or 800).

[0079] In one embodiment, when both the at least one main camera C0-C3 and the at least one accessory camera C4-C5 capture the object images, the processors P1 and P2 independently calculate the object pose according to the object images. And the processor P2 transmits the object pose calculated by the processor P2 to the processor P1. Then, the processor P1 generates a fusion pose according to the object pose calculated by the processor P2 and the object pose calculated by the processor P1.

[0080] Embodiments of the present invention provide an object tracking system and method equipped with modularized accessory cameras of an additional device. The accessory cameras are pluggable as needed. An additional processor in the accessory device could help the head-mounted display device search for and track the controller. In this way, the object tracking system and methods achieve the effect of better distributed computing power. Furthermore, the object tracking system and methods can flexibly add at least one camera module (i.e., accessory camera) to an existing module (i.e., head-mounted display device) to expand the field of view of the head-mounted display device. The object tracking range (e.g., controller tracking range) can also be expanded.

[0081] Although the invention has been illustrated and described with respect to one or more implementations, equivalent changes and modifications will occur or will be apparent to others skilled in the art based on a reading and understanding of this specification and the accompanying drawings. Furthermore, while a particular feature of the invention may have been disclosed with respect to only one of several implementations, such a feature may be combined with one or more other features of the other implementations as may be desired and advantageous for a particular or specialized application.

Claims

[1] Object tracking system, comprising: a head-mounted display (HMD) comprising: a first processor configured to transmit search information for searching for an object during a search phase; an accessory device connected to the HMD device, comprising: at least one accessory camera; and a second processor connected to the at least one accessory camera and configured to receive the search information and determine whether one of the at least one accessory cameras captured an object image during the search phase; wherein, when the second processor determines that at least one of the at least one accessory camera has captured the object image, the second processor transmits notification information to the first processor and the first processor enters a tracking phase and transmits request information to the second processor; when the second processor receives the request information, the second processor performs one of the following actions: Transmitting the object image to the first processor, wherein the first processor calculates an object pose according to the object image; and Calculating the object position based on the object image and transmitting the object position to the first processor. [2] The object tracking system of claim 1, wherein the HMD device further comprises at least one main camera and the first processor determines whether one of the at least one main camera has captured an object image during the search phase. [3] The object tracking system of claim 1, wherein the notification information includes camera information, the camera information indicates that a particular accessory camera has captured the object image, and the first processor continuously tracks or calculates the object pose according to the object image captured by the particular accessory camera in the tracking phase. [4] The object tracking system according to claim 1, wherein the second processor transmits handover information to the HMD device, the handover information indicating that another specific accessory camera is capturing the object image, and the first processor performs a handover process to continuously track or calculate the object position of the object image captured by the other accessory camera in the tracking phase. [5] The object tracking system according to claim 1, wherein the request information includes a cropping command, and when the second processor receives the request information, the second processor crops the object image to obtain a partial image of the object image still containing the object and transmits the partial image to the first processor, and the first processor calculates the object pose according to the partial image. [6] The object tracking system of claim 2, wherein the accessory device is a front cover device for the HMD device, the main cameras include a left side camera, a right side camera, a left front camera, and a right front camera, and the at least one accessory camera includes a camera arranged on the top side of the front cover device and another camera arranged on the bottom side of the front cover device. [7] Object tracking methods, comprising: Transmission of search information for the search for an object by a first processor during a search phase; Receiving the search information and using a second processor to determine whether one of at least one accessory camera has captured an object image in the search phase; wherein, when the second processor determines that at least one of the at least one accessory camera has captured the object image, the second processor transmits notification information to the first processor and the first processor enters a tracking phase and transmits request information to the second processor; when the second processor receives the request information, the second processor performs one of the following actions: Transmitting the object image to the first processor, wherein the first processor calculates an object pose according to the object image; and Calculating the object position based on the object image and transmitting the object position to the first processor. [8] The object tracking method of claim 7, wherein the HMD device further comprises at least one main camera and the first processor determines whether one of the at least one main camera has captured an object image during the search phase. [9] The object tracking method according to claim 7, wherein the notification information includes camera information, the camera information indicating that a specific accessory camera has captured the object image, and the first processor continuously tracks or calculates the object pose according to the object image captured by the specific accessory camera in the tracking phase. [10] The object tracking method according to claim 7, wherein the second processor transmits handover information to the HMD device, and the handover information indicates that another specific accessory camera captures the object image, and the first processor performs a handover process to continuously track or calculate the object position of the object image captured by the other accessory camera in the tracking phase. [11] The object tracking method according to claim 7, wherein the request information includes a cropping command, and when the second processor receives the request information, the second processor crops the object image to obtain a partial image of the object image still containing the object and transmits the partial image to the first processor, and the first processor calculates the object pose according to the partial image. [12] The object tracking method according to claim 8, wherein the accessory device is a front cover device for the HMD device, the main cameras include a left side camera, a right side camera, a left front camera, and a right front camera, and the at least one accessory camera includes a camera arranged on the top side of the front cover device and another camera arranged on the bottom side of the front cover device.

Images