Object tracking method and system and non-transient computer readable storage medium

Abstract

The invention provides an object tracking method, an object tracking system and a non-transitory computer readable storage medium. The object tracking method is suitable for an object tracking system used for tracking a target object, and comprises the following steps: obtaining first spatial relationship information between an electronic device and an anchor point object through first tracking equipment; obtaining second spatial relationship information between the anchor point object and the target object through a second tracking device; and calculating, by the information processor, spatial information of the target object relative to the electronic device according to the first spatial relationship information and the second spatial relationship information. By obtaining the first spatial relationship information between the electronic device and the anchor point object and the second spatial relationship information between the anchor point object and the target object, even if the target object is shielded, the object tracking system and method can calculate the spatial information of the target object relative to the electronic device in real time. Therefore, the object tracking system and method have the advantages of being wider in tracking view field and the like.

Description

Technical Field

[0001] The present disclosure relates to a method and a system, in particular to an object tracking method and system. Background Art

[0002] In the field of object tracking, some related technologies may use some tracking technologies, such as trackers, infrared light-emitting diodes, optical codes (e.g., Quick Response Matrix Code (QR code)), etc. However, these related technologies may face some problems when tracking tiny objects (e.g., scalpels). For example, most trackers are not suitable for tiny objects due to physical limitations (e.g., weight, size, etc.). Infrared light-emitting diodes and optical codes are easily blocked or soiled, thus losing their effectiveness. Some related technologies design and create a specific environment for tracking, which may not be widely applicable to various different sites and may cost a large amount of money. Therefore, it is necessary to provide new methods to solve the above problems. Summary of the Invention

[0003] One aspect of the present disclosure is an object tracking method. The object tracking method is applicable to an object tracking system for tracking a target object, and includes: obtaining a first spatial relationship information between an electronic device and an anchor object through a first tracking device; obtaining a second spatial relationship information between the anchor object and the target object through a second tracking device; and calculating a spatial information of the target object relative to the electronic device based on the first spatial relationship information and the second spatial relationship information by an information processor.

[0004] In some embodiments, the first tracking device includes a first tracking device and a first trackable device, and the object tracking method further includes: respectively disposing the first tracking device and the first trackable device on the anchor object and the electronic device; or respectively disposing the first tracking device and the first trackable device on the electronic device and the anchor object.

[0005] In some embodiments, obtaining the first spatial relationship information between the electronic device and the anchor object through the first tracking device includes: calculating a first attitude data of the first trackable device relative to the first tracking device by interacting with the first trackable device through the first tracking device, where the first spatial relationship information includes the first attitude data.

[0006] In some embodiments, the second tracking device includes a second tracking means and a second trackable means, and the object tracking method further includes: respectively disposing the second tracking means and the second trackable means on the anchor object and the target object; or respectively disposing the second tracking means and the second trackable means on the target object and the anchor object.

[0007] In some embodiments, obtaining the second spatial relationship information between the anchor object and the target object by the second tracking device includes: interacting with the second trackable means through the second tracking means to calculate a second attitude data of the second trackable means relative to the second tracking means, where the second spatial relationship information includes the second attitude data.

[0008] In some embodiments, the object tracking method further includes: setting the first tracking means and the second tracking means disposed on the anchor object to have a fixed spatial relationship information; setting the first tracking means and the second trackable means disposed on the anchor object to have the fixed spatial relationship information; setting the first trackable means and the second tracking means disposed on the anchor object to have the fixed spatial relationship information; or setting the first trackable means and the second trackable means disposed on the anchor object to have the fixed spatial relationship information.

[0009] In some embodiments, calculating, by the information processor, the spatial information of the target object relative to the electronic device according to the first spatial relationship information and the second spatial relationship information includes: calculating a third attitude data of the target object relative to the electronic device according to a first attitude data of the first trackable means relative to the first tracking means, a second attitude data of the second trackable means relative to the second tracking means, and a fixed attitude data, where the second tracking means is set to have the fixed attitude data relative to the first tracking means or the first trackable means, or the second trackable means is set to have the fixed attitude data relative to the first tracking means or the first trackable means.

[0010] In some embodiments, the object tracking method further includes: determining the anchor object in a real-world environment where the electronic device and the target object are located, where the anchor object has a fixed position in the real-world environment.

[0011] In some embodiments, the object tracking method further includes: determining another electronic device in a real-world environment where the electronic device and the target object are located as the anchor object, where the another electronic device is movable in the real-world environment and is used to position itself in the real-world environment.

[0012] In some embodiments, the electronic device includes a display, and the object tracking method further includes: providing, by the electronic device via the display, immersive content including virtual reality content indicating the spatial information.

[0013] Another aspect of the present disclosure is an object tracking system. The object tracking system is used to track a target object and includes an electronic device, a first tracking device, a second tracking device, and an information processor. The first tracking device is used to obtain a first spatial relationship information between the electronic device and an anchor object. The second tracking device is used to obtain a second spatial relationship information between the anchor object and the target object. The information processor is used to calculate a spatial information of the target object relative to the electronic device based on the first spatial relationship information and the second spatial relationship information.

[0014] Another aspect of the present disclosure is a non-transitory computer-readable storage medium having a computer program for executing an object tracking method, where the object tracking method is applicable to an object tracking system for tracking a target object and includes: obtaining, by a first tracking device, a first spatial relationship information between an electronic device and an anchor object; obtaining, by a second tracking device, a second spatial relationship information between the anchor object and the target object; and calculating, by an information processor, a spatial information of the target object relative to the electronic device based on the first spatial relationship information and the second spatial relationship information.

[0015] In summary, by obtaining the first spatial relationship information between the electronic device and the anchor object and the second spatial relationship information between the anchor object and the target object, even if the target object is blocked, the object tracking system and method of the present disclosure can immediately calculate the spatial information of the target object relative to the electronic device. In other words, the object tracking system and method of the present disclosure have advantages such as a wider tracking field of view. Description of the Drawings

[0016] Figure 1 It is a block diagram of an object tracking system according to some embodiments of the present disclosure.

[0017] Figure 2 It is a flowchart of an object tracking method according to some embodiments of the present disclosure.

[0018] Figure 3 It is a schematic diagram of a scenario where the object tracking system according to some embodiments of the present disclosure is applied to a medical system.

[0019] Figure 4 It is a schematic diagram of an immersive content according to some embodiments of the present disclosure.

[0020] Figure 5 Schematic diagram of another scenario where the object tracking system according to some embodiments of the present disclosure is applied to a medical system.

[0021] Figure 6 Block diagram of another object tracking system according to some embodiments of the present disclosure.

[0022] Symbol description:

[0023] 10, 50: Electronic device

[0024] 11, 15: Target object

[0025] 13: Anchor object

[0026] 20: First tracking device

[0027] 30: Second tracking device

[0028] 31: Scalpel

[0029] 33: Operating lamp

[0030] 35: Operating room

[0031] 37: Forceps

[0032] 40: Information processor

[0033] 60: Third tracking device

[0034] 100, 600: Object tracking system

[0035] 101: Display

[0036] 200: Object tracking method

[0037] 201: First tracking device

[0038] 203: First trackable device

[0039] 300: Medical system

[0040] 301: Second tracking device

[0041] 303: Second trackable device

[0042] 331: Columnar part

[0043] 601: Third tracking device

[0044] 603: Third trackable device

[0045] CI: Immersive content

[0046] R1: Lesion

[0047] S201 - S203: Operations

[0048] U1, U3: Doctors

[0049] U2: Patients

[0050] V1: Virtual reality content Detailed implementation manners

[0051] Examples are given below in conjunction with the accompanying drawings for detailed description. However, the specific examples described are only used to explain this case and do not limit this case. The description of the structure and operation is not used to limit the execution order. Any structure recombined by components, and the resulting device with equivalent functions, are all within the scope covered by this disclosure content.

[0052] Regarding the "coupling" or "connection" used in this article, it can refer to two or more components making direct physical or electrical contact with each other, or making indirect physical or electrical contact with each other. It can also refer to two or more components operating or acting on each other.

[0053] Please refer to Figure 1 , Figure 1 FIG. is a block diagram of an object tracking system 100 according to some embodiments of the present disclosure. In some embodiments, the object tracking system 100 includes a first tracking device 20, a second tracking device 30, and an information processor 40. The object tracking system 100 is used to track a target object 11 in a real - world environment (not shown in Figure 1 ).

[0054] In some practical applications, at least one electronic device 10, the first tracking device 20 and the second tracking device 30 of the object tracking system 100 are all in the real - world environment with the target object 11. The information processor 40 is communicatively coupled to the electronic device 10, the first tracking device 20, and the second tracking device 30. Therefore, the information processor 40 can be set in the real - world environment or other places. It should be noted that based on an anchor object 13 determined in the real - world environment, the spatial information of the target object 11 relative to the electronic device 10 can be obtained through the operations of the first tracking device 20, the second tracking device 30, and the information processor 40.

[0055] The operations of the first tracking device 20, the second tracking device 30, and the information processor 40 will be described in detail below in conjunction with an object tracking method 200. Figure 2 FIG. is a flowchart of an object tracking method 200 according to some embodiments of the present disclosure. The object tracking method 200 is applicable to Figure 1 the object tracking system 100. In some embodiments, as Figure 2As shown, the object tracking method 200 includes a plurality of operations S201 to S203.

[0056] In operation S201, the first tracking device 20 obtains a first spatial relationship information between the electronic device 10 and the anchor object 13. In some embodiments, as Figure 1 shown, the first tracking device 20 includes a first tracking device 201 and a first trackable device 203. Specifically, the first tracking device 201 is disposed on the electronic device 10, and the first trackable device 203 is disposed on the anchor object 13. With such an arrangement, the first tracking device 20 will be able to obtain the first spatial relationship information between the electronic device 10 and the anchor object 13.

[0057] Continuing Figure 1 from the above embodiments, the first tracking device 201 on the electronic device 10 can interact with the first trackable device 203 on the anchor object 13, so as to calculate a first attitude data POS1 of the first trackable device 203 relative to the first tracking device 201 as the first spatial relationship information. For example, the first tracking device 201 and the first trackable device 203 can be implemented by an infrared camera and an infrared light emitting diode respectively. The first trackable device 203 can emit infrared light, and the first tracking device 201 can sense the infrared light emitted by the first trackable device 203, so as to deduce the first attitude data POS1 through, for example, a perspective n-point algorithm. It should be noted that the first attitude data POS1 calculated by the first tracking device 201 on the electronic device 10 can indicate the six degrees of freedom of the anchor object 13 relative to the electronic device 10.

[0058] It should be understood that the arrangements of the first tracking device 201 and the first trackable device 203 are not limited to Figure 1 the arrangements in the above embodiments. In some embodiments, the first tracking device 201 is disposed on the anchor object 13, and the first trackable device 203 is disposed on the electronic device 10. With such an arrangement, the first attitude data POS1 of the first trackable device 203 relative to the first tracking device 201 can also be calculated as the first spatial relationship information. Further, the first attitude data POS1 calculated by the first tracking device 201 on the anchor object 13 can indicate the six degrees of freedom of the electronic device 10 relative to the anchor object 13.

[0059] From the descriptions of the above embodiments, it can be seen that in some embodiments, one of the first tracking device 201 and the first trackable device 203 is disposed on the anchor object 13, and the other of the first tracking device 201 and the first trackable device 203 is disposed on the electronic device 10.

[0060] In operation S202, the second tracking device 30 obtains a second spatial relationship information between the anchor object 13 and the target object 11. In some embodiments, as Figure 1 shown, the second tracking device 30 includes a second tracking device 301 and a second trackable device 303. Specifically, the second tracking device 301 is disposed on the anchor object 13, and the second trackable device 303 is disposed on the target object 11. With such an arrangement, the second tracking device 30 will be able to obtain the second spatial relationship information between the anchor object 13 and the target object 11.

[0061] Continuing Figure 1 the embodiments of, the second tracking device 301 on the anchor object 13 can interact with the second trackable device 303 on the target object 11, so as to calculate a second attitude data POS2 of the second trackable device 303 relative to the second tracking device 301 as the second spatial relationship information. For example, the second tracking device 301 and the second trackable device 303 can be implemented by one or more ultrasonic receivers and an ultrasonic transmitter respectively. The second trackable device 303 can emit ultrasonic waves, and the second tracking device 301 can sense the ultrasonic waves emitted by the second trackable device 303, so as to derive the second attitude data POS2 by, for example, the three-point method. It should be noted that the second attitude data POS2 calculated by the second tracking device 301 on the anchor object 13 can indicate the six degrees of freedom of the target object 11 relative to the anchor object 13.

[0062] It should be understood that the arrangements of the second tracking device 301 and the second trackable device 303 are not limited to Figure 1 the arrangements in the embodiments. In some embodiments, the second tracking device 301 is disposed on the target object 11, and the second trackable device 303 is disposed on the anchor object 13. With such an arrangement, the second attitude data POS2 of the second trackable device 303 relative to the second tracking device 301 can also be calculated as the second spatial relationship information. Further, the second attitude data POS2 calculated by the second tracking device 301 on the target object 11 can indicate the six degrees of freedom of the anchor object 13 relative to the target object 11.

[0063] From the descriptions of the above embodiments, it can be seen that in some embodiments, one of the second tracking device 301 and the second trackable device 303 is disposed on the anchor object 13, and the other of the second tracking device 301 and the second trackable device 303 is disposed on the target object 11.

[0064] In operation S203, the information processor 40 calculates the spatial information of the target object 11 relative to the electronic device 10 based on the first spatial relationship information and the second spatial relationship information. In some embodiments, the information processor 40 receives the first spatial relationship information and the second spatial relationship information from the first tracking device 20 and the second tracking device 30 respectively, so as to calculate the spatial information of the target object 11 relative to the electronic device 10.

[0065] Continuing with the embodiments of operation S203, in some further embodiments, as Figure 1 shown, the information processor 40 is communicatively coupled to the first tracking device 201 and the second tracking device 301, and receives the first attitude data POS1 (i.e., the first spatial relationship information) and the second attitude data POS2 (i.e., the second spatial relationship information) from the first tracking device 201 and the second tracking device 301 respectively. Further, the information processor 40 can pre-store or access a fixed attitude data POSF from a storage device (not shown in the figure), where the storage device can be implemented by a volatile memory, a non-volatile memory, or both. It should be noted that this fixed attitude data POSF indicates the fixed six degrees of freedom of the second tracking device 301 relative to the first trackable device 203. Accordingly, the information processor 40 can use the first attitude data POS1, the second attitude data POS2, and the fixed attitude data POSF to calculate a third attitude data POS3 of the target object 11 relative to the electronic device 10 as the spatial information of the target object 11 relative to the electronic device 10. The third attitude data POS3 can indicate the six degrees of freedom of the target object 11 relative to the electronic device 10.

[0066] From the description of the above embodiments, it can be seen that the object tracking method 200 should not be limited to including multiple operations S201 - S203. For example, in some embodiments, before operation S201, the object tracking method 200 further includes an operation of respectively setting the first tracking device 201 and the first trackable device 203 on the anchor object 13 and the electronic device 10 or an operation of respectively setting the first tracking device 201 and the first trackable device 203 on the electronic device 10 and the anchor object 13.

[0067] In some embodiments, before operation S201, the object tracking method 200 further includes an operation of respectively setting the second tracking device 301 and the second trackable device 303 on the anchor object 13 and the target object 11 or an operation of respectively setting the second tracking device 301 and the second trackable device 303 on the target object 11 and the anchor object 13.

[0068] In some embodiments, before operation S201, the object tracking method 200 further includes setting the first tracking device 201 and the second tracking device 301 disposed on the anchor object 13 to have a fixed spatial relationship information, setting the first tracking device 201 and the second trackable device 303 disposed on the anchor object 13 to have fixed spatial relationship information, setting the first trackable device 203 and the second tracking device 301 disposed on the anchor object 13 to have fixed spatial relationship information, or setting the first trackable device 203 and the second trackable device 303 disposed on the anchor object 13 to have fixed spatial relationship information. For example, in the embodiment of Figure 1 the second tracking device 301 is set to have a fixed six degrees of freedom (i.e., fixed pose data POSF) relative to the first trackable device 203, and the fixed six degrees of freedom of the second tracking device 301 relative to the first trackable device 203 can be referred to as fixed spatial relationship information. It should be understood that in some embodiments, the first trackable device 203 can be set to have a fixed six degrees of freedom relative to the second tracking device 301, and thus the fixed six degrees of freedom of the first trackable device 203 relative to the second tracking device 301 can be referred to as fixed spatial relationship information.

[0069] In some embodiments, before operation S201, the object tracking method 200 further includes determining the operation of the anchor object 13 in the real-world environment where the electronic device 10 and the target object 11 are located. In some further embodiments, the anchor object 13 has a fixed position in the real-world environment, which will be further described in the following paragraphs in conjunction with Figure 3 further explanation.

[0070] Please refer to Figure 3 , Figure 3 FIG. 14 is a schematic diagram of a scenario in which an object tracking system 100 according to some embodiments of the present disclosure is applied to a medical system 300. In some embodiments, the medical system 300 includes a scalpel 31, an operating lamp 33, and an operating room 35. The medical system 300 is typically used by at least one medical staff (e.g., doctor U1, etc.) to perform medical treatment. For example, in the operating room 35, doctor U1 uses the scalpel 31 to perform surgery on patient U2 under the illumination provided by the operating lamp 33.

[0071] When the object tracking system 100 is to be applied to the medical system 300, the target object 11 to be tracked by the object tracking system 100 should be determined. Also, the anchor object 13 should be determined to enhance the ability of the object tracking system 100 to track the target object 11. For example, in Figure 3In the embodiment, the scalpel 31 is determined as the target object 11, and a columnar portion 331 of the surgical lamp 33 fixed in the operating room 35 (i.e., having a fixed position in the real-world environment) is determined as the anchor object 13. Further, in Figure 3 the embodiment, the electronic device 10 is a head-mounted device and is mounted on the head of the doctor U1 to notify the doctor U1 of the spatial information of the scalpel 31 (i.e., the target object 11) relative to the electronic device 10.

[0072] In Figure 3 the embodiment, the settings of the first tracking device 20 and the second tracking device 30 are the same as those in Figure 1 the embodiment. That is, the first tracking device 201 is disposed on the electronic device 10, the first trackable device 203 and the second tracking device 301 are disposed on the columnar portion 331 (i.e., the anchor object 13), and the second trackable device 303 is disposed on the scalpel 31 (i.e., the target object 11). In addition, the information processor 40 is disposed on the electronic device 10 and can transmit the calculation result (e.g., spatial information) to the electronic device 10. Calculating the spatial information of the scalpel 31 relative to the electronic device 10 is similar to Figure 1 and 2 the description in the embodiment, so it will not be elaborated here.

[0073] In some further embodiments, as Figure 1 shown, the electronic device 10 includes a display 101. The electronic device 10 can be communicatively coupled to the information processor 40 via at least one communicator (not shown in the figure) in the electronic device 10, so that at least one controller / processor (not shown in the figure) in the electronic device 10 receives the spatial information of the target object 11 relative to the electronic device 10 (e.g., the third attitude data POS3). Then, the controller / processor in the electronic device 10 can use the display 101 to provide, for example, Figure 3 the doctor U1 in with an immersive content CI that shows the spatial information of the scalpel 31 (i.e., the target object 11) relative to the electronic device 10.

[0074] Continuing with the above embodiment where the electronic device 10 provides the immersive content CI to the doctor U1 via the display 101, the immersive content CI can be an Augmented Reality (AR) environment. Specifically, the augmented reality environment can enhance the real-world environment (i.e., the operating room 35) directly seen by the doctor U1 with virtual reality objects (which cannot be directly seen by the doctor U1 in the real-world environment), which will be further described in the following paragraphs in conjunction with Figure 4 further.

[0075] Please refer to Figure 4 , Figure 4Schematic diagram of immersive content CI shown in some embodiments according to the present disclosure. In Figure 4 the embodiment, the immersive content CI provided by the electronic device 10 worn by the doctor U1 shows a scalpel 31, a lesion R1 (or wound, incision, etc.) on the patient U2, and a virtual reality content V1 (i.e., a virtual reality object). It should be understood that the direct visibility of the scalpel 31 by the doctor U1 may be blocked by, for example, body fluids (such as blood), organs, etc. during the operation. It is worth noting that the virtual reality content V1 can indicate the spatial information of the scalpel 31 relative to the electronic device 10 in a known manner. In this way, even if the direct visibility of the scalpel 31 by the doctor U1 is blocked, the doctor U1 can clearly know the position of the scalpel 31 through the virtual reality content V1 in the immersive content CI.

[0076] In Figure 3 and 4 the embodiment, the immersive content CI provided by the electronic device 10 is an augmented reality environment, but the present disclosure is not limited thereto. In some embodiments, the immersive content CI may be a virtual reality (VR) environment or a mixed reality (MR) environment. Specifically, the mixed reality environment simulates the real-world environment and enables virtual reality objects to interact with the simulated environment. In addition, in the embodiment where the electronic device 10 is a head-mounted device, the doctor U1 wearing the electronic device 10 can control the virtual reality object (e.g., the virtual reality content V1) in the immersive content CI by operating at least one controller (not shown in the figure) or by making hand or eye movements. Specifically, at least one controller can be wirelessly connected to the electronic device 10, and hand or eye movements will be sensed and recognized by the electronic device 10 via at least one camera (not shown in the figure).

[0077] Continuing with the embodiment where the first tracking device 201 is implemented by an infrared camera, in some further embodiments, the first tracking device 201 provided on the electronic device 10 has a field of view (FOV) aligned with the first trackable device 203 provided on the anchor object 13. With such a setting, it will be possible to ensure that the first tracking device 201 can track the first trackable device 203.

[0078] In the above embodiments, as Figure 3 shown, a rigid object (i.e., the columnar part 331 of the surgical lamp 33) in the real-world environment is determined as the anchor object 13, but the present disclosure is not limited thereto. For example, in some embodiments, a movable object in the real-world environment is determined as the anchor object 13, which will be further described in the following paragraphs in conjunction with Figure 5 further explanation.

[0079] Please refer to Figure 5 , Figure 5 which is a schematic diagram of another scenario where the object tracking system 100 according to some embodiments of the present disclosure is applied to the medical system 300. As Figure 5 shown, another electronic device 50 (for example, another head-mounted device) installed on the head of another doctor U3 is determined as the anchor object 13, and the first tracking device 203 and the second tracking device 301 are both disposed on the electronic device 50. Therefore, the anchor object 13 in these embodiments is movable in the real-world environment (for example, the operating room 35).

[0080] In addition, each of the electronic device 10 and the electronic device 50 can position itself in the real-world environment through some vision-based positioning technologies (for example, simultaneous localization and mapping technology, etc.), which helps the object tracking system 100 to track the target object 11. For example, the electronic device 10 can obtain its pose in a map of the real-world environment through the vision-based positioning technology, and the electronic device 50 can obtain its pose in another map of the real-world environment through the vision-based positioning technology. Also, the electronic device 10 and the electronic device 50 can share data (for example, the map established by the electronic device 10, the map established by the electronic device 50, the pose of the electronic device 10, the pose of the electronic device 50, etc.) with the information processor 40. Through the data shared by the electronic device 10 and the electronic device 50, the information processor 40 can obtain or calculate another spatial relationship information between the electronic device 10 and the electronic device 50 (for example, the pose data of the electronic device 50 relative to the electronic device 10, the pose data of the electronic device 10 relative to the electronic device 50, etc.). When the first tracking device 20 accidentally fails to obtain the first spatial relationship information between the electronic device 10 and the anchor object 13 (that is, the electronic device 50), the information processor 40 can use another spatial relationship information between the electronic device 10 and the electronic device 50 (which is obtained based on the data generated by the vision-based positioning technology) and the second spatial relationship information between the anchor object 13 and the target object 11 to calculate the spatial information of the target object 11 relative to the electronic device 10.

[0081] It should be understood that the object tracking system of the present disclosure is not limited to only tracking the target object 11. For example, please refer to Figure 6 , Figure 6 which is a block diagram of another object tracking system 600 shown according to some embodiments of the present disclosure. In the Figure 6 embodiment of, the object tracking system 600 is used to track the target object 11 and another target object 15. For example, the target object 15 can be in the medical system 300 by Figure 5A pair of pliers 37 held by doctor U3. In order to track both the target object 11 and the target object 15, the object tracking system 600 includes an electronic device 10, a first tracking device 20, a second tracking device 30, an information processor 40, and a third tracking device 60.

[0082] In some embodiments, the third tracking device 60 is used to obtain a third spatial relationship information between the anchor object 13 and the target object 15. Specifically, the third tracking device 60 includes a third tracking device 601 and a third trackable device 603. The third tracking device 601 is disposed on the anchor object 13, and the third trackable device 603 is disposed on the target object 15. The third tracking device 601 on the anchor object 13 can interact with the third trackable device 603 on the target object 15, so as to calculate the attitude data (not shown in the figure) of the third trackable device 603 relative to the third tracking device 601 as the third spatial relationship information. For example, the third tracking device 601 and the third trackable device 603 can be implemented by an electromagnetic field generator and an electromagnetic sensor respectively. It should be noted that the attitude data calculated by the third tracking device 601 on the anchor object 13 can indicate the six degrees of freedom of the target object 15 relative to the anchor object 13.

[0083] Further explanation, Figure 6 The information processor 40 can calculate the spatial information of the target object 15 relative to the electronic device 10 based on the first spatial relationship information obtained by the first tracking device 20 and the third spatial relationship information obtained by the third tracking device 60. Specifically, in Figure 6 the embodiment, the third tracking device 601 is set to have a fixed six degrees of freedom relative to the first trackable device 203. The information processor 40 calculates the attitude data (not shown in the figure) of the target object 15 relative to the electronic device 10 as the spatial information of the target object 15 relative to the electronic device 10 based on the first attitude data POS1, the attitude data calculated by the third tracking device 601 on the anchor object 13, and the fixed attitude data indicating the fixed six degrees of freedom of the third tracking device 601 relative to the first trackable device 203. Figure 6 The operations of the first tracking device 20 and the second tracking device 30 in Figure 1 are the same as those in the embodiment, so they will not be elaborated here.

[0084] It should be understood that the spatial information of the target object 15 relative to the electronic device 10 can be displayed in the immersive content CI together with the spatial information of the target object 11 relative to the electronic device 10.

[0085] It should also be understood that the settings of the third tracking device 601 and the third trackable device 603 are not limited to Figure 6The settings in the embodiments are limitations. In some embodiments, the third tracking device 601 is disposed on the target object 15, and the third trackable device 603 is disposed on the anchor object 13. With such a setting, the attitude data indicating the six degrees of freedom of the anchor object 13 relative to the target object 15 can also be calculated as the third spatial relationship information.

[0086] From the descriptions of the first tracking device 20, the second tracking device 30, and the third tracking device 60, it can be seen that the first tracking device 20, the second tracking device 30, and the third tracking device 60 can be implemented by three different tracking techniques respectively. However, the present disclosure is not limited thereto. In some embodiments, the first tracking device 20, the second tracking device 30, and the third tracking device 60 can be implemented by the same tracking technique respectively, or can be implemented by three tracking techniques that are partially the same and partially different. In the above embodiments, the tracking techniques include infrared rays, electromagnetic fields, videos / images, ultrasonic waves, lidar, sonar, structured light sources, time-of-flight ranging systems, etc. Since the object tracking system 100 provides a high degree of flexibility for the tracking techniques among the electronic device 10, the anchor object 13, and the target object 11, it is convenient to apply the object tracking system 100 to various situations. For example, the user of the object tracking system 100 can select a tracking technique suitable for the type of the target object 11 (e.g., the scalpel 31), and can select another tracking technique different from the one tracking technique used between the anchor object 13 and the target object 11 for the electronic device 10. If the other tracking technique has the advantage of low cost, the cost of applying the object tracking system 100 can be reduced.

[0087] It should be understood that the object tracking system of the present disclosure is not limited to tracking the target object 11 only based on the anchor object 13. For example, in some embodiments, the number of the anchor objects 13 can be greater than 1.

[0088] In the above embodiments, the information processor 40 can be implemented by a central processing unit, an application specific integrated circuit, a microprocessor, a system on a chip, or other suitable processing circuits. Further, the information processor 40 can be an independent device, or can be integrated into the electronic device 10 and / or the anchor object 13.

[0089] In some further embodiments, each tracking device (e.g., the first tracking device 20, the second tracking device 30, and the third tracking device 60) in the object tracking system of the present disclosure can be paired with at least one inertial measurement unit to increase the precision of the spatial relationship information between any two of the electronic device 10, at least one anchor object 13, and at least one target object 11.

[0090] Moreover, in some further embodiments, the medical system 300 can label and track the lesion R1 using various known medical techniques (e.g., contrast agents, X-rays, etc.). The six degrees of freedom of the lesion R1 relative to an origin (not shown in the figure) defined by the medical system 300 can be provided to the information processor 40. With such a setting, the information processor 40 converts the six degrees of freedom of the lesion R1 into the spatial information of the lesion R1 relative to the electronic device 10 in a known manner. In addition, the immersive content CI provided by the electronic device 10 can indicate the spatial information of at least one target object 11 and the spatial information of the lesion R1. In this way, the doctor U1 wearing the electronic device 10 will not need to watch other monitors in the operating room 35 during the operation, thus improving the operation efficiency and reducing the fatigue of the doctor U1.

[0091] As can be seen from the above embodiments of the present disclosure, by obtaining the first spatial relationship information between the electronic device 10 and the anchor object 13 and the second spatial relationship information between the anchor object 13 and the target object 11, even if the target object 11 is occluded, the object tracking system and method of the present disclosure can instantaneously calculate the spatial information of the target object 11 relative to the electronic device 10. In other words, the object tracking system and method of the present disclosure have advantages such as a wider tracking field of view.

[0092] Furthermore, in some practical applications, by using ultrasonic tracking technology (e.g., the second tracking device 30) to obtain the second spatial relationship information between the anchor object 13 and the target object 11, the object tracking system and method of the present disclosure can avoid the problems of occlusion or smudging of optical trackers (e.g., infrared light-emitting diodes, optical codes, etc.).

[0093] The method of the present disclosure can exist in the form of program code. The program code can be included in a physical medium, such as a floppy disk, a CD, a hard disk, or any other transient or non-transient computer-readable storage medium. When the program code is loaded and executed by a computer, this computer becomes a device for implementing the method. The program code can also be transmitted through some transmission media, such as wires or cables, through optical fibers, or through any other transmission form. When the program code is received, loaded, and executed by a computer, this computer becomes a device for implementing the method. When implemented on a general-purpose processor, the program code combines with the processor to provide a unique device that operates similarly to application-specific logic circuits.

[0094] Although the present disclosure has been disclosed as above in embodiments, it is not intended to limit the present disclosure. Those of ordinary skill in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to that defined by the appended claims.

Claims

1. An object tracking method, characterized in that: An object tracking system for tracking a target object, comprising: Obtaining first spatial relationship information between an electronic device and an anchor object through a first tracking device; Obtaining second spatial relationship information between the anchor object and the target object through a second tracking device; as well as An information processor is used to calculate spatial information of the target object relative to the electronic device according to the first spatial relationship information and the second spatial relationship information.

2. The object tracking method according to claim 1, characterized in that: The first tracking device includes a first tracking device and a first trackable device, and the object tracking method further includes: placing the first tracking device and the first trackable device on the anchor object and the electronic device respectively; or The first tracking device and the first trackable device are respectively disposed on the electronic device and the anchor object.

3. The object tracking method according to claim 2, characterized in that: Obtaining the first spatial relationship information between the electronic device and the anchor object through the first tracking device includes: The first tracking device interacts with the first trackable device to calculate a first posture data of the first trackable device relative to the first tracking device, wherein the first spatial relationship information includes the first posture data.

4. The object tracking method according to claim 2, wherein: The second tracking device includes a second tracking device and a second trackable device, and the object tracking method further includes: placing the second tracking device and the second trackable device on the anchor object and the target object respectively; or The second tracking device and the second trackable device are respectively disposed on the target object and the anchor object.

5. The object tracking method according to claim 4, characterized in that: Obtaining the second spatial relationship information between the anchor object and the target object by the second tracking device includes: The second tracking device interacts with the second trackable device to calculate a second posture data of the second trackable device relative to the second tracking device, wherein the second spatial relationship information includes the second posture data.

6. The object tracking method according to claim 4, characterized in that: Also includes: The first tracking device and the second tracking device disposed at the anchor point object are set to have fixed spatial relationship information; Setting the first tracking device and the second trackable device disposed at the anchor point object to have the fixed spatial relationship information; Setting the first trackable device and the second tracking device disposed at the anchor object to have the fixed spatial relationship information; or The first trackable device and the second trackable device disposed at the anchor point object are set to have the fixed spatial relationship information.

7. The object tracking method according to claim 4, characterized in that: Calculating the spatial information of the target object relative to the electronic device by the information processor according to the first spatial relationship information and the second spatial relationship information includes: A third posture data of the target object relative to the electronic device is calculated based on a first posture data of the first trackable device relative to the first tracking device, a second posture data of the second trackable device relative to the second tracking device, and a fixed posture data, wherein the second tracking device is set to have the fixed posture data relative to the first tracking device or the first trackable device, or the second trackable device is set to have the fixed posture data relative to the first tracking device or the first trackable device.

8. The object tracking method according to claim 1, wherein: Also includes: The anchor object is determined in a real-world environment where the electronic device and the target object are located, wherein the anchor object has a fixed position in the real-world environment.

9. The object tracking method according to claim 1, wherein: Also includes: Another electronic device in a real-world environment where the electronic device and the target object are located is determined to be the anchor object, wherein the other electronic device is movable in the real-world environment and is used to locate itself in the real-world environment.

10. The object tracking method according to claim 1, wherein: The electronic device includes a display, and the object tracking method further includes: An immersive content including a virtual reality content indicating the spatial information is provided through the display by the electronic device.

11. An object tracking system, characterized in that: Used to track a target object, and includes: an electronic device; A first tracking device, used to obtain first spatial relationship information between the electronic device and an anchor object; a second tracking device, used to obtain second spatial relationship information between the anchor object and the target object; as well as An information processor is used to calculate spatial information of the target object relative to the electronic device according to the first spatial relationship information and the second spatial relationship information.

12. A non-transitory computer-readable storage medium, characterized in that: A computer program is provided for executing an object tracking method, wherein the object tracking method is applicable to an object tracking system for tracking a target object, and comprises: Obtaining first spatial relationship information between an electronic device and an anchor object through a first tracking device; Obtaining second spatial relationship information between the anchor object and the target object through a second tracking device; as well as An information processor is used to calculate spatial information of the target object relative to the electronic device according to the first spatial relationship information and the second spatial relationship information.

Images