Optical tracking system with infrared data transmission

Abstract

An optical tracking system may include an optical sensor (e.g., an infrared (IR) camera) coupled to an illuminator (e.g., an infrared light-emitting diode (LED)). The optical tracking system can track an object having retroreflective markers. Light from the illuminator is reflected by the retroreflective markers, which can be detected by the optical sensor. In some cases, the object may include an emitter (e.g., an IR LED) such that active light emitted from the object is directly detected by the optical sensor. The optical sensor generates data based on the detected light and transmits this data to a controller. The controller can calculate position data (e.g., indicating the relative two-dimensional position between the object and the optical sensor). The controller can transmit the position data optically (e.g., using infrared light) to the object.

Description

Technical Field

[0001] 〔Cross - Reference to Related Applications〕 This application claims priority and the benefit thereof to U.S. Provisional Patent Application No. 63 / 447,206, entitled "OPTICAL TRACKING SYSTEM WITH DATA TRANSMISSION VIA INFRARED," filed on February 21, 2023, the entire disclosure of which is hereby incorporated by reference in its entirety for all purposes.

Background Art

[0002] An automatic tracking system can have various applications (e.g., tracking an object, issuing an alarm based on the tracked object) in a specific location (e.g., amusement park, building, parking lot). One type of automatic tracking system can be an optical tracking system that utilizes various optical sensors to detect light reflected from one or more tracked objects. The optical tracking system can generate position data indicating the respective positions of one or more tracked objects based on the detected light.

[0003] This section is for introducing readers to various aspects of the technology that may be related to the various aspects of the technology described and / or claimed hereinafter. This discussion is considered to be helpful in showing readers the background situation and facilitating a better understanding of the various aspects of the present disclosure. Therefore, these descriptions should not be construed as an admission of prior art and should be understood to be read from the above perspective.

Summary of the Invention

[0004] The following summarizes several embodiments that fall within the same scope as the subject matter of the original claims. These embodiments are not intended to limit the scope of the disclosure, but rather to outline some of the disclosed embodiments. In practice, the disclosure may include a variety of forms that are similar to or different from the embodiments shown below.

[0005] In one embodiment, the system includes an optical sensor configured to detect an object within an area. The system also includes a controller configured to receive first data indicating a first position of the object from the optical sensor, second data indicating a second position of the optical sensor, calculate position data based on the first and second positions, and transmit the position data to the object via an optical signal.

[0006] In one embodiment, the optical tracking system includes an optical sensor configured to detect a first object and a second object within an area. The system also includes a controller configured to receive first data from the optical sensor indicating a first position of the first object. The controller receives second data from the optical sensor indicating a second position of the second object, and also receives third data indicating further positions of the optical sensor. The controller calculates first position data for the first object based on the first and further positions, and calculates second position data for the second object based on the second and further positions. The controller also commands a light emitter to transmit the first position data to the first object via a first optical signal, and commands the light emitter to transmit the second position data to the second object via a second optical signal.

[0007] In one embodiment, the method includes receiving location data from an optical sensor configured in one or more processors to detect light from an object in an area. The method also includes calculating position data indicating the relative position between the object and the area based on the location data via one or more processors. The method further includes transmitting the position data to the object via an optical transmitter.

[0008] A better understanding of these and other features, aspects and advantages of this disclosure will be gained by reading the following detailed description while referring to the attached drawings, which indicate the same parts throughout with the same reference numerals. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram of an optical tracking system according to an embodiment of the present disclosure, which includes a controller coupled to a plurality of optical sensors equipped with light-emitting elements that facilitate tracking of objects having retroreflectors. [Figure 2] This is a schematic diagram of an optical tracking system according to an embodiment of the present disclosure, which includes a controller coupled to a plurality of optical sensors that detect light emitted from a light-emitting element coupled to an object. [Figure 3] This is a schematic diagram of an optical tracking system according to an embodiment of the present disclosure, including a controller coupled to an object worn by a guest (e.g., a head-mounted display [HMD]). [Figure 4] This is a schematic diagram of an optical tracking system, including one optical sensor having a light-emitting element that facilitates tracking of multiple objects, according to an embodiment of the present disclosure. [Figure 5] This is an example of an application in an amusement park using the optical tracking system shown in Figure 4, according to an embodiment of the present disclosure. [Figure 6] This is a flowchart illustrating a method for tracking an object using an optical tracking system according to embodiments of the present disclosure. [Modes for carrying out the invention]

[0010] The following describes one or more specific embodiments. For the sake of brevity, this specification does not describe all features of these embodiments. Furthermore, the development of any such implementation found in any engineering or design project requires numerous implementation-specific decisions to achieve the developer's specific objectives, such as compliance with system-related and business-related constraints, which may vary by implementation. Moreover, while such development efforts can be complex and time-consuming, they are routine design, fabrication, and manufacturing activities for those skilled in the art who benefit from this disclosure.

[0011] When describing elements of the various embodiments of this disclosure, the articles “a,” “an,” “the,” and “said” mean that there are one, two, or more of these elements. The terms “comprising,” “including,” and “having” are intended to be comprehensive and mean that there may be further elements other than those listed. Hereinafter, one or more specific embodiments of the embodiments described herein will be described. In order to describe these embodiments concisely, not all features of the actual implementation may be described herein. Furthermore, in developing any such implementation found in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developer’s specific objectives, such as compliance with system-related and business-related constraints, which may vary by implementation. Moreover, while such development efforts can be complex and time-consuming, they are routine design, fabrication, and manufacturing activities for those skilled in the art who will benefit from this disclosure.

[0012] In many situations, optical tracking systems, such as motion capture systems, can be useful. One common type of motion capture system uses a series of infrared (IR) cameras, each containing a lens and an IR light source facing the same direction as the lens. The object(s) to be tracked have retroreflective markers. The IR light source emits IR light, which is reflected by the retroreflective markers and returned to the lenses of the IR cameras. All IR cameras transmit their images to a controller, which calculates the relative positions of all IR cameras and the objects(s) in three-dimensional (3D) space. The controller transmits data to other devices using wired, wireless, and / or radio frequencies for use in larger systems.

[0013] One drawback of this type of motion capture system is that, if the tracked (one or multiple) objects are to receive their own position data, the (one or multiple) objects need to have a wired, wireless, or radio frequency connection. Wired connections can restrict the movement of the (one or multiple) objects. Furthermore, limitations of wireless data transmission via radio frequencies can result in long delays between image capture and the (one or multiple) objects receiving position data. Keeping delays low is particularly important in certain applications such as augmented reality (AR) and virtual reality (VR). For example, this is now recognized as particularly important for head-mounted displays (HMDs) that are worn by guests and configured to project AR images for guest visualization (e.g., projecting AR images in conjunction with the real world environment based on the HMD's position data relative to the real world environment in space).

[0014] Furthermore, this type of motion capture system introduces additional delays due to the time required to transfer images to the controller and process the position of retroreflective markers in the software. Additionally, transmitting data to another system or back to the (single or multiple) objects themselves using radio frequencies requires careful calibration of the mesh network to avoid interference and occlusion from the physical structure.

[0015] This disclosure relates to an optical tracking system capable of tracking one or more objects (e.g., portable devices including wearable devices, vehicles) and providing location data to one or more objects. The optical tracking system can be used in any of a variety of environments, such as amusement parks, theaters, restaurants, workplaces, residences, parking lots and / or storage locations. In one embodiment, the optical tracking system can utilize various optical sensors (e.g., cameras, photodetectors, LiDAR) that detect light (e.g., infrared light, visible light) reflected or emitted from one or more objects. The optical tracking system can analyze the received signals from the optical sensors to generate location data (e.g., relative positions between the optical sensors and one or more objects, relative positions between one or more objects and the environment) and transmit the location data to one or more objects. In this way, each of the one or more objects can recognize its respective position relative to the environment. It is advantageous that the optical tracking system can transmit location data to one or more objects via optical signals such as IR light.

[0016] In one embodiment, the optical tracking system may include optical sensors, each coupled to one or more illuminators (e.g., light emitters, light-emitting diodes [LEDs], arrays of LEDs, or other types of actively powered illuminators) capable of emitting light (e.g., IR light, visible light) to illuminate one or more objects. Each of the one or more objects may have a retroreflective marker that can reflect some of the light emitted from the one or more illuminators. The optical sensors can detect this reflected light and generate a signal indicating the position of that object among one or more objects in the environment. Each optical sensor of the optical sensor may transmit this signal to a controller (e.g., communicatively coupled to or built into a camera), which can process the signal to generate position data (including, for example, the relative position between the object and the optical sensor, and the relative position between the object and the environment, which can be derived from or determined from known relationships between the relative position between the object and the optical sensor and the optical sensor and the environment). Furthermore, the controller can transmit position data to an object via optical signals (e.g., one or more illuminators, modulated optical signals encoding position data). The object can then recognize its own position (e.g., position relative to optical sensors and / or the environment) by receiving these optical signals using a photodetector.

[0017] In one embodiment, one or more objects may have coupled emitters (e.g., LEDs, arrays of LEDs, or other types of actively powered illuminators, such as flashing LEDs that emit bright, but short, sudden or intermittent light) that emit light (e.g., infrared light, visible light). The emitted light can be detected by optical sensors in the vicinity of one or more objects. When each of the one or more optical sensors receives light emitted from a particular object among the one or more objects, it transmits a signal indicating the location of this particular object to a controller, which can process the signal to generate location data for the particular object. The controller can instruct one or more illuminators in the environment to emit optical signals containing the location data. The objects can receive the location data via photodetectors.

[0018] Optical tracking systems can offer several advantages, including improved data transmission and reduced signal interference. For example, since an optical sensor detects one or more objects using a line of sight to one or more objects, this same line of sight can be used to transmit data to one or more objects using one or more illuminators in the optical sensor. Using light for data transmission also reduces the likelihood of interference from other devices, such as mobile phones (compared to, for example, using radio frequencies for data transmission).

[0019] Based on the above, Figure 1 is a schematic diagram of an embodiment of an optical tracking system 12, which includes a controller (e.g., a control system, an electrical controller) 16 coupled to an optical sensor 18 equipped with a light-emitting element 24 for tracking an object 20 located within an area 14 (e.g., an amusement park, a theater, a restaurant, a workplace, a residential area, a parking lot, a storage area). As shown in the figure, the controller 16 can be located in a control room and coupled to the optical sensor 18 in a manner that allows communication (e.g., wired or wireless).

[0020] In one embodiment, the optical sensor 18 may include a camera operating with visible light (e.g., a wavelength range of 380 nanometers to 700 nanometers), infrared (IR) light (e.g., a wavelength range of 780 nanometers to 1 millimeter), or other suitable light. In another embodiment, the optical sensor 18 may include other optical sensors and optical tracking devices, such as a LiDAR sensor or IR sensor used to detect distance based on the detection of light (e.g., modulated light), an optical sensor that provides artificial vision for object recognition and tracking (e.g., to the controller 16), or any combination thereof.

[0021] The optical sensor 18 may include a light-emitting element 24 that emits light (e.g., visible light or IR light) to illuminate an object 20. For example, optical sensors 18A, 18B, 18C, and 18D may each include emitters 24A, 24B, 24C, and 24D, respectively. Each emitter 24A, 24B, 24C, and 24D may include one or more light-emitting units (e.g., IR light or visible light-emitting diodes [LEDs]) that generate light (e.g., infrared light or visible light) to illuminate an object 20 in area 14. For example, a light-emitting element 24A coupled to optical sensor 18A may emit light 30 to illuminate an object 20, and a light-emitting element 24B coupled to optical sensor 18B may similarly emit light 60 to illuminate an object 20.

[0022] The object 20 can be any optically trackable object, such as a portable object configured to be worn or carried by a guest. For example, the portable object can include any handheld device and / or wearable device (e.g., a head-mounted display [HMD], a mobile phone, a tablet, a souvenir, a toy, a band, a cane). However, it should be understood that the object 20 can also include other types of objects, such as furniture and vehicles (e.g., a ride vehicle in an amusement park). The object 20 can include a retroreflector 36 (e.g., at least one of a retroreflective marker, a reflecting device, or a reflecting surface) that retroreflects radiation (e.g., light) to a light emitter 24 (e.g., with limited scattering). In some cases, the retroreflector 36 can be coupled to or incorporated into (woven, painted, machined, molded) the object 20 (e.g., via a fastener such as an adhesive, a screw, a bolt).

[0023] The optical sensor 18 (e.g., optical sensors 18A - 18D) can detect light retroreflected from the object 20 (e.g., light emitted from the light emitter 24 and reaching the object 20). For example, the optical sensor 18A can detect the reflected light 40 (e.g., a part of the light 30 emitted from the light emitter 24A and retroreflected from the retroreflector 36 of the object 20). Similarly, the optical sensor 18B can detect the reflected light 70 (e.g., a part of the light 60 emitted from the light emitter 24B and retroreflected from the retroreflector 36 of the object 20).

[0024] The optical sensor 18 can generate a signal indicating the position of the object 20 after detecting the reflected light (e.g., lights 40 and 70). For example, the optical sensor 18A can generate a signal 42A based on the detected light 40 (e.g., based on the arrival time, direction, light intensity, and / or other attributes such as frequency and / or polarization of the light 40). The signal 42A can indicate the relative position between the object 20 and the optical sensor 18A. The optical sensor 18B can generate a signal 42B based on the detected light 70 (e.g., based on the arrival time, direction, light intensity, and / or other attributes). The signal 42B can indicate the relative position between the object 20 and the optical sensor 18B. Similarly, other optical sensors (e.g., 18C, 18D) can also generate corresponding signals (e.g., 42C, 42D) based on the detected light reflected from the retroreflector 36 of the object 20.

[0025] In certain embodiments, each of the signals (e.g., 42A - 42D) can include additional information. For example, the signal 42A can include information indicating the position of the optical sensor 18A (e.g., the relative position between the optical sensor 18A and the area 14 according to a local coordinate system established for the area 14, an identifier of the optical sensor 18A that links to the position of the optical sensor 18A). Such information can be used to calculate the position of the object 20 with respect to the area 14 based on the signals (e.g., 42A - 42D).

[0026] The optical sensors 18 (e.g., 18A-18D) can transmit signals (e.g., 42A-42D) to the controller 16 (e.g., via a communication line such as an electrical cable 46 or a fiber optic cable). The controller 16 may include one or more processors 82, a memory device 84, and a communication component 86. In some embodiments, the controller 16 may include further components such as receivers and transmitters that receive or transmit data (e.g., via radio frequencies, optical frequencies, other communication frequencies, or any combination thereof). In some embodiments, the controller 16 may include input / output devices and / or displays that allow a user (e.g., a guest) associated with the object 20 to interact with specific data (e.g., a location map, an image).

[0027] In one embodiment, each of the optical sensors 18 can transmit a simplified version of the data indicating the position of the object 20 to the controller 16. For example, an optical sensor 18A (including a camera) can reduce the amount of data transmitted and increase the overall bandwidth of the optical tracking system 12 by performing basic blob detection to identify where the retroreflector 36 is located in the captured image frame and transmitting only the image frame containing the identified retroreflector 36 to the controller 16.

[0028] The processor 82 can process instructions executed within the controller 16. The processor 82 may include a (single or multiple) single-threaded processor, a (single or multiple) multi-threaded processor, or both. The processor 82 can process instructions stored in memory 84. The processor 82 may also include a (single or multiple) hardware-based processor, each containing one or more cores. The processor 82 may include a (single or multiple) general-purpose processor, a (single or multiple) dedicated processor, or both. The processor 82 may include one or more general-purpose microprocessors, one or more application-specific processors (ASICs), one or more field-programmable gate arrays (FPGAs), or any combination thereof. For example, a dedicated processor may include a (single or multiple) artificial intelligence processor designed based on machine learning and artificial neural networks. The (single or multiple) artificial intelligence processor can read various positional data related to object 20, other objects near object 20, and optical sensor 18, and perform calculations based on the positional data. The processor 82 can be communicatively coupled to other internal components (such as memory 84, communication component 86, input / output devices, and displays).

[0029] Memory 84 can be any preferred manufactured article that can function as a medium for storing processor executable code and data, etc. These manufactured articles can represent a computer-readable medium (e.g., any preferred form of memory or storage) that can store processor executable code used by the processor 82 to perform the techniques of this disclosure. Applications used herein can include any preferred computer software or program that can be installed on the controller 16 and run by the processor 82. Memory device 84 can represent a non-temporary computer-readable medium (e.g., any preferred form of memory or storage) that can store processor executable code used by the processor 82 to perform the various techniques described herein. For example, memory device 84 can include random access memory (RAM), read-only memory (ROM), rewritable non-volatile memory such as flash memory, a hard drive, and / or an optical disc. "Non-temporary" simply indicates that the medium is tangible and not a signal.

[0030] The communication component 86 can be a wireless or wired communication component that facilitates communication between the controller 16 and other devices (e.g., optical sensor 18) over a network. For example, the communication component 86 can enable the controller 16 to acquire data from various data sources such as the optical sensor 18, one or more databases (e.g., optical sensor location database, controller location database, map database), user devices (e.g., object 20, HMD, smartphone, tablet), and vehicle systems (e.g., driving systems on or inside a vehicle). The communication component 86 can receive notifications and transmit them to the user devices and / or vehicle systems. The communication component 32 can use various communication protocols such as Open Database Connectivity (ODBC), TCP / IP protocol, Distributed Relational Database Architecture (DRDA) protocol, Database Change Protocol (DCP), HTTP protocol, other suitable current or future protocols, or combinations thereof.

[0031] The processor 82 can be configured to receive signals 42A to 42D transmitted from optical sensors 18A to 18D (for example, via a communication component 86). In response, the processor 82 can calculate positional data related to the object 20. For example, based on signal 42A, the processor 82 can calculate the positional data of the object 20 relative to the optical sensor 18A. Calculating the positional data may include analyzing the attributes of the light 40, calculating the relative distance and orientation between the object 20 and the optical sensor 18A, extracting the coordinates 22 of the optical sensor 18A, and calculating the coordinates 26 of the object 20 (for example, the relative coordinates between the object 20 and area 14).

[0032] The coordinate 22 may include coordinate data of the optical sensor 18A, such as Cartesian coordinates (XC, YC, and ZC) measured with respect to a Cartesian coordinate system including axes 52, 53, and 56. The coordinate data may also include azimuth data such as YAWC, PITCHC, and ROLLC indicating the orientation of the optical sensor 18A. The coordinate 22 can be relative coordinates (e.g., a local coordinate system established for area 14) or non-relative coordinates (e.g., a global coordinate system such as Global Positioning System [GPS] coordinates).

[0033] Coordinate 26 can include coordinate data of object 20 relative to optical sensor 18. For example, coordinate 26 can include the Cartesian coordinates (XO, YO, and ZO) of object 20 relative to optical sensor 18A. The coordinate data of object 20 can also include orientation data such as YAWO, PITCHO, and ROLLO, indicating the orientation of object 20 relative to optical sensor 18A.

[0034] In one embodiment, the signals (e.g., 42A-42D) may not include the positions of the optical sensors (e.g., 18A-18D). For example, the controller 16 may store the coordinates of each optical sensor 18 (e.g., via a memory device 84). In such a case, the controller 16 can use the stored positions of the optical sensors (e.g., 18A-18D) to calculate the coordinates 26 of the object 20.

[0035] After the controller 16 calculates the coordinates 26 of the object 20, it can transmit the coordinates 26 to the optical sensor 18 (for example, via the communication component 88). For example, the controller 16 can transmit a portion of the coordinates 26 (for example, the portion corresponding to the relative coordinates of the object 20 with respect to the optical sensor 18A) to the optical sensor 18A. The optical sensor 18A can respond by transmitting an optical signal to the object 20. The optical signal may include the relative coordinates of the object 20 with respect to the optical sensor 18A. The optical sensor 18A can generate an optical signal using an optical transmitter 44A and transmit the optical signal via light 48 (for example, infrared light or visible light, a modulated optical signal encoding position data). The object 20 may include a photodetector 54 configured to receive light 48 that carries position data (for example, the relative coordinates of the object 20 with respect to the optical sensor 18A). The object 20 can use the position data to recognize its position relative to the optical sensor 18A and / or area 14. It should be understood that the optical sensor 18A can generate an optical signal using the emitter 24A and transmit the optical signal via light 48 (e.g., infrared light or visible light, a modulated optical signal encoding position data). In such cases, the optical sensor 18A may include an emitter 24A that emits light 30 to be reflected back from the retroreflector 36 of the object 20, as well as light 48 that provides position data to the object 20 (for example, the optical sensor 18A does not include an independent optical transmitter such as the optical transmitter 44A shown in Figure 1, and the emitter 24A is the optical transmitter 44A, or operates as the optical transmitter 44A). In practice, any of the optical sensors 18A, 18B, 18C, or 18D can track the object 20 and provide position data to the object 20 using their respective emitters 24A, 24B, 24C, or 24D.

[0036] Similarly, the controller 16 can transmit different parts of the coordinates 26 (for example, parts corresponding to the relative coordinates of object 20 with respect to optical sensor 18B) to optical sensor 18B. In response, optical sensor 18B can transmit different optical signals to object 20. These different optical signals may include the relative coordinates of object 20 with respect to optical sensor 18B. Optical sensor 18B can generate optical signals using an optical transmitter 44B (or emitter 24B) and transmit the optical signals via light 78 (for example, infrared or visible light). Object 20 can receive the light 78 conveying position data (for example, the relative coordinates of object 20 with respect to optical sensor 18B) using a photodetector 54. Object 20 can use the position data to recognize its position relative to optical sensor 18A and / or area 14. Furthermore, it should be understood that the controller 16 can determine the coordinates 26 of object 20 (for example, relative to area 14) and instruct one or more of the optical sensors (e.g., 18A and / or 18B) to provide the coordinates 26 of object 20 to object 20 via their respective optical signals (e.g., light 48 and / or 78).

[0037] In one embodiment, the light-emitting element 24 on or within the optical sensor 18 may include a digital projector or similar directional light sources. Such directional light sources can increase the overall bandwidth of the optical tracking system 12 by enabling each of the optical sensors 18 to transmit position data to a specific object being tracked (e.g., object 20) without each of the optical sensors 18 transmitting the same data to other objects being tracked (e.g., object 20).

[0038] In one embodiment, object 20 may further process and / or utilize position data received via the photodetector 54 using specific devices (e.g., devices on object 20, and / or connection devices such as a vehicle controller mounted on a vehicle and wired to object 20). For example, object 20 could be an HMD that uses coordinates 26 to extract an image and displays it so that the image, overlaid on the real-world environment within area 14, appears collaboratively to a guest wearing the HMD (e.g., the image is overlaid so that it appears to blend into the real-world environment). In another example, position data can trigger effects on object 20 (e.g., light, sound, tactile) (e.g., object 20 is programmed to output specific effects based on object 20's position and / or orientation). In yet another example, object 20 could be a mobile phone case that supports a mobile phone and is communicably coupled to the mobile phone (e.g., wired or wirelessly). In this case, the mobile phone case transmits location data to the mobile phone, and an application on the mobile phone can use the location data to display relevant information or enable interaction with area 14 via input on the mobile phone. As another example, the location data can enable object 20 and / or a connecting device to acquire and / or determine enhanced location data, such as a location map showing the relative position of object 20 to other objects in area 14 (e.g., other HMDs in a vehicle). Thus, the optical tracking system 12 can enable certain coordinated events (e.g., theme park events involving multiple guests and / or multiple rider cars) based on the relative positions of multiple objects, including object 20.

[0039] Although the controller 16 and optical sensor 18 are described as being communicatively coupled to each other via an electrical cable 46, in some embodiments, the controller 16 and optical sensor 18 can also be communicatively coupled to each other via radio frequency signals, optical signals (e.g., using visible light or IR light). In some embodiments, the controller 16 or optical sensor 18 can transmit position data to the object 20 using wired signals or radio frequency signals in addition to optical signals (e.g., via lights 48 and 78). Such further data transmission can function as backup communication (e.g., when the default communication is malfunctioning, being interfered with, or being disrupted).

[0040] In some cases, more or fewer devices (e.g., tracking devices such as optical sensors 18) or components (e.g., light emitters 24) can be implemented in the optical tracking system 12 using different methods. For example, in one embodiment, the light emitter 24 can be coupled to one or more structures within area 14 so that the optical sensor 18 does not include the light emitter 24. As a result, the light emitter 24 can be hidden from the view of guests in area 14. For example, an object 20 can be placed on a tabletop, inside a tabletop, or covering a tabletop so that it is visible to guests in area 14, with at least a portion of the object 20 having a light receiver 54 (e.g., the base of the object 20) exposed in the space below the tabletop. Then, the light emitter 24 hidden under the tabletop can transmit position data so that the light receiver 54 can detect it. In one embodiment, a single optical sensor 18 can be used to track the object 20 in area 14. In one embodiment, several devices or components can be a combination of the object to be tracked and the tracking device.

[0041] Based on the above, Figure 2 is a schematic diagram of an embodiment of an optical tracking system 12 including a controller 16 coupled to an optical sensor 18 (e.g., optical sensors 18A to 18D). The optical sensor 18 is configured to receive light 48 and / or 78 emitted from a light emitter 24F coupled to an object 20. The optical sensor 18 may not include a light emitter (e.g., light emitter 24 shown in Figure 1) that emits light 48 and / or 78 (e.g., visible light or IR light) to illuminate the object 20. Instead, the object 20 may have a light emitter 24F that can emit light 48 and / or 78 (e.g., visible light or IR light) detectable by the optical sensor 18. For example, the light emitter 24F may be coupled to and / or incorporated into the object 20.

[0042] The light-emitting element 24F can emit light 48 and / or 78 (e.g., visible light or IR light). A portion of this light (e.g., light 110) can reach the optical sensor 18A. The optical sensor 18A can detect the light 110 and generate a signal indicating the position of the object 20 relative to the optical sensor 18A. For example, the optical sensor 18A can generate a signal 42A based on the detected light 110 (e.g., based on the arrival time, direction, light intensity, and / or other attributes of the light 110). The signal 42A can indicate the relative position between the object 20 and the optical sensor 18A.

[0043] Different portions of light (e.g., light 120) can reach the optical sensor 18B. The optical sensor 18B can detect light 120 and generate a signal indicating the position of object 20 relative to the optical sensor 18B. For example, the optical sensor 18B can generate a signal 42B based on the detected light 120 (e.g., based on the arrival time, direction, light intensity, and / or other attributes of light 120). The signal 42B can indicate the relative position between object 20 and optical sensor 18B. Similarly, other optical sensors (e.g., 18C, 18D) can also generate corresponding signals (e.g., 42C, 42D) based on detected light emitted from the light emitter 24F of object 20.

[0044] As illustrated with reference to Figure 1, the processor 82 receives signals 42A to 42D transmitted from optical sensors 18A to 18D (e.g., via the communication component 86) and calculates position data (e.g., coordinates 26) associated with the object 20. After calculating the coordinates 26 of the object 20, the controller 16 can transmit the coordinates 26 to one or more of the optical sensors 18 (e.g., via the communication component 88). In response, one or more of the optical sensors 18 can transmit an optical signal to the object 20 (e.g., via the corresponding optical transmitter 44) via the corresponding light (e.g., light 48 or 78). The object 20 can use the light receiver 54 to receive the light 48 and / or 78 that convey the position data (e.g., the coordinates 26 of the object 20). As described herein, the object 20 can further process and / or utilize the position data based on the position data received via the light receiver 54 using certain devices. For example, positional data can trigger effects on object 20 (e.g., light, sound, touch) (for example, object 20 is programmed to output specific effects based on its position and / or orientation).

[0045] Figure 3 is a schematic diagram of an embodiment of an optical tracking system 12, including a controller 16 coupled to an object (e.g., a head-mounted display (HMD) 160) worn by a guest 170 in area 14. Similar to Figure 2, the optical sensor 18 may not include a light emitter 24 that emits light (e.g., visible light or IR light) to illuminate the HMD 160 and the guest 170. As shown, the HMD 160 may include a light emitter 24F (e.g., as an integrated device) that can emit light (e.g., visible light or IR light) detectable by the optical sensor 18. As shown, the controller 16 may be part of the HMD 160.

[0046] For example, a portion of the light emitted from the light-emitting element 24F within the HMD 160 (e.g., light 164) can reach the optical sensor 18A. The optical sensor 18A can detect light 164 and generate a signal indicating the position of the HMD 160 (and guest 170) relative to the optical sensor 18A. The optical sensor 18A can generate a signal based on the detected light 164 (e.g., based on the arrival time, direction, light intensity, and / or other attributes of light 164). This signal can indicate the relative position between the HMD 160 and the optical sensor 18A.

[0047] The optical sensor 18A can transmit this signal to the HMD 160 as an optical signal (encoded into light 168, for example, via the optical transmitter 44A). The HMD 160 can receive light 168 using the light receiver 54. Furthermore, the HMD 160 can use the controller 16 to calculate position data associated with the HMD 160 (for example, coordinates 26 relative to the optical sensor 18A and / or area 14). Coordinates 26 may include coordinate data of the HMD 160 relative to the optical sensor 18A and / or area 14.

[0048] Similarly, the optical sensor 18B can transmit different signals to the HMD 160 as different optical signals (e.g., encoded in light 178 via the optical transmitter 44B). The different optical signals can indicate the relative position between the HMD 160 and the optical sensor 18B. The HMD 160 can receive the light 178 using the light receiver 54. Furthermore, the HMD 160 can use the controller 16 to calculate position data related to the HMD 160 (e.g., coordinates 26 relative to the optical sensor 18B and / or area 14). Coordinates 26 may include coordinate data of the HMD 160 relative to the optical sensor 18B and / or area 14.

[0049] The HMD160 can use location data (e.g., coordinates 26) to provide information to the guest 170 and / or adjust its operation. For example, the HMD160 can display the location of the guest 170 within area 14. As another example, the HMD160 can use location data to display a specific image (e.g., an AR image and / or a VR image) that the guest 170 sees based on the location data (e.g., in coordination with effects within area 14 and / or the real-world environment within area 14). Specifically, the HMD160 can display a corresponding image in coordination with effects such as sound effects, visual effects and / or haptic effects within area 14 (e.g., an AR or VR image of a dragon in coordination with heat from a heat source, or an AR image of a bird that appears to be perched on a roof in coordination with a roof background image on the display within the real-world environment). In this way, the optical tracking system 12 can enhance the experience within area 14 by enabling or improving certain coordinated events (e.g., theme park events involving guest 170 and other guests) based on the relative positions of multiple guests, including guest 170.

[0050] As described herein, the HMD160 may include an AR device that provides an enhanced interactive version of the real-world environment (e.g., Area 14) achieved by overlaying and / or coordinating images displayed on the HMD160 with other digital visual elements, sounds, and / or other sensory stimuli (e.g., via haptic technology). In such cases, the HMD160 can further enhance the guest's experience through specific coordinated events and effects by utilizing the position of the HMD160 to provide a more precise overlay display of images onto the real-world environment.

[0051] Although the controller 16 is described as a standalone part of the HMD 160, in some embodiments, the functions and / or components of the controller 16 may be incorporated into the HMD 160. For example, in some embodiments, the HMD 160 may not include the controller 16. Instead, any suitable processor in the HMD 160, and / or any suitable processor communicably coupled to the HMD 160 (e.g., via a wired or wireless connection), may perform similar functions, such as calculating positional data (e.g., coordinates 26) related to the HMD 160.

[0052] In one embodiment, each of the optical sensors 18 can transmit a simplified version of the data indicating the position of the HMD 160 to the HMD 160 for further processing. For example, an optical sensor 18A (including a camera) can reduce the amount of data transmitted and improve communication bandwidth by performing basic blob detection to identify image frames with light from the light emitter 24F, and then transmitting only the identified image frames to the HMD 160.

[0053] Figure 4 is a schematic diagram of an embodiment of an optical tracking system 12 that includes one optical sensor 18H with a light-emitting element 24H for tracking multiple objects. For example, the multiple objects to be tracked may include HMD 212 worn by guest 210 and HMD 222 worn by guest 220. As shown in the figure, the controller 16 is integrated with the optical sensor 18H.

[0054] For example, the optical sensor 18H may include one or more wide-angle view cameras capable of capturing images of multiple objects at different locations in area 14. In some cases, the optical sensor 18H may be mounted on or inside one or more flying objects (e.g., drones 204) or structures (e.g., signal towers 208). Such flying objects or structures can provide the optical sensor 18H with a better field of view to improve object tracking within area 14. In some embodiments, object 20 may include a flying object (e.g., drone 204). The drone 204 may include an aerial vehicle that is autonomously controlled (e.g., according to a programmed flight path) and / or remotely controlled (e.g., according to control inputs provided by a remote operator).

[0055] As shown in the figure, the optical sensor 18H can track multiple objects (e.g., HMDs 212 and 222) simultaneously. For example, the optical sensor 18H (including, for example, one or more wide-angle view cameras) can identify and track each object by detecting a known marker pattern or shape associated with a retroreflector 36 coupled to the corresponding object. Alternatively, in one embodiment, the optical sensor 18H can also identify and track each object (e.g., HMDs 212 and 222) by reading light containing active IR pulses (e.g., having different frequencies from each object being tracked) emitted from each emitter 24.

[0056] HMD212 may have a retroreflector 36A that can reflect light (e.g., IR light or visible light) emitted from the light-emitting element 24H of the optical sensor 18H (e.g., coupled to the frame, visor, and / or band of HMD212). Similarly, HMD222 may have a retroreflector 36B that can reflect light (e.g., IR light or visible light) emitted from the light-emitting element 24H of the optical sensor 18H (e.g., coupled to the frame, visor, and / or band of HMD222). It should be understood that retroreflectors 36A and 36B may each represent or include multiple different retroreflectors.

[0057] For example, the retroreflector 36A can reflect a portion of the light 214 emitted from the light emitter 24H. The reflected light (e.g., light 216) can reach the optical sensor 18H. The optical sensor 18H can detect the light 216 and generate a first signal indicating the position of the HMD 212 relative to the optical sensor 18H. The optical sensor 18H can generate a first signal based on the detected light 216. The first signal can indicate the relative position between the guest 210 and the optical sensor 18H and / or area 14.

[0058] The controller 16 (for example, the processor 82 of the controller 16) can be configured to receive a first signal transmitted from the optical sensor 18H (for example, via a communication component 86). In response, the controller 16 can calculate position data related to the guest 210 and / or HMD 212. For example, the controller 16 can calculate position data of the HMD 212 relative to the optical sensor 18H based on the first signal. Calculating the position data may include analyzing the attributes of the light 216, calculating the relative distance and orientation between the HMD 212 and the optical sensor 18H, extracting the coordinates 22 of the optical sensor 18H, and calculating the coordinates 26 of the HMD 212 (for example, the relative coordinates between the optical sensor 18H and the HMD 212, and / or the relative coordinates between the HMD 212 and area 14). The coordinates 26 of the HMD 212 may include the coordinate data of the HMD 212 relative to the optical sensor 18H. In one embodiment, the controller 16 can directly calculate the coordinates 26 of the HMD 212 relative to area 14 using stored location data (e.g., the coordinates 22 of the optical sensor 18H).

[0059] After calculating the coordinates 26 of the HMD 212, the controller 16 can transmit the coordinates 26 of the HMD 212 to the optical sensor 18H (for example, via the communication component 88). In response, the optical sensor 18H can transmit a first optical signal to the HMD 212 via light 218 (for example, via the optical transmitter 44H or the light emitter 24H). The first optical signal may include the coordinates 26 of the HMD 212. The HMD 212 can receive the light 218 carrying the first optical signal using the photodetector 54A.

[0060] Similarly, the retroreflector 36B on the HMD222 can reflect a portion of the light 230 emitted from the light emitter 24H. The reflected light (e.g., light 234) can reach the optical sensor 18H (e.g., from different angles of incidence relative to light 216). The optical sensor 18H can detect the light 234 and generate a second signal indicating the position of the HMD222 relative to the optical sensor 18H. The optical sensor 18H can generate a second signal based on the detected light 234. The second signal can indicate the relative position between the guest 220 and the optical sensor 18H and / or area 14.

[0061] The controller 16 (for example, the processor 82 of the controller 16) can be configured to receive a second signal transmitted from the optical sensor 18H. In response, the controller 16 can calculate position data related to the guest 220 and / or HMD 222. For example, the controller 16 can calculate position data of the HMD 222 relative to the optical sensor 18H based on the second signal. Calculating the position data may include analyzing attributes related to the light 234, calculating the relative distance and orientation between the HMD 222 and the optical sensor 18H, extracting the coordinates 22 of the optical sensor 18H, and calculating the coordinates 26 of the HMD 222 (for example, the relative coordinates between the optical sensor 18H and the HMD 222, and / or the relative coordinates between the HMD 222 and area 14). The coordinates 26 of the HMD 222 may include the coordinate data of the HMD 222 relative to the optical sensor 18H. In one embodiment, the controller 16 can directly calculate the coordinates 26 of the HMD 212 relative to area 14 using stored location data (e.g., the coordinates 22 of the optical sensor 18H).

[0062] After the controller 16 calculates the coordinates 26 of the HMD 222, it can transmit the coordinates 26 of the HMD 222 to the optical sensor 18H (for example, via the communication component 88). In response, the optical sensor 18H can transmit a second optical signal to the HMD 222 via light 238 (for example, via the optical transmitter 44H or the light emitter 24H). The second optical signal may include the coordinates 26 of the HMD 222. The HMD 222 can receive the light 238 carrying the second optical signal using the photodetector 54B.

[0063] The optical tracking system 12 described herein with reference to Figures 1 to 4 includes tracking at least one object (e.g., object 20) using a plurality of optical sensors 18, or tracking multiple objects (e.g., HMDs 212 and 222) using at least one optical sensor (e.g., optical sensor 18H), but in some embodiments, different features can be implemented in the optical tracking system 12. For example, in some embodiments, the optical tracking system 12 may include simultaneously tracking multiple objects in an area 14 using a plurality of optical sensors 18. For example, data can be transmitted to a given object (e.g., HMD 212 or HMD 222) using only the light within its field of view. In addition to or instead of this, an optical sensor can also transmit data (e.g., via a directional light source and / or an encoded optical signal for a specific object) only for a given object within its field of view.

[0064] Based on the above, Figure 5 shows an example application 250 in an amusement park using the optical tracking system 12. Guests 210 and 220 can wear HMDs 212 and 222, respectively, when riding a ride vehicle 256 (e.g., traveling along a path 258) within area 14 (e.g., an amusement park). As described herein, the HMD 212 may have a retroreflector 36A that can reflect a portion of the light 214 emitted from the light emitter 24H. The reflected light (e.g., light 216) can reach the optical sensor 18H, which can detect the light 216 and generate a first signal indicating the relative position between the HMD 212 and the optical sensor 18H. Furthermore, the optical sensor 18H can generate a second signal indicating the relative position between the HMD 222 and the optical sensor 18H based on the light 234 reflected from the retroreflector 36B coupled to the HMD 222. For clarity, Figure 5 does not show light 234 and retroreflector 36B.

[0065] The controller 16 (for example, the processor 82 of the controller 16) can be configured to receive a first signal and a second signal from the optical sensor 18H (for example, via a communication component 86). In response, the controller 16 can calculate position data related to the HMDs 212 and 222. The position data may include the coordinates of the HMDs 212 and 222, respectively. After calculating the coordinates of the HMDs 212 and 222, the controller 16 can transmit the coordinates of the HMDs 212 and 222 to the optical sensor 18H (for example, via a communication component 88), and the optical sensor 18H may transmit a first optical signal to the HMD 212 via light 218 (for example, via an optical transmitter 44H or light emitter 24H) and a second optical signal to the HMD 212 via light 238 (not shown in Figure 5 for clarity). The first and second optical signals may include the relative coordinates of the HMDs 212 and 222, respectively, with respect to the optical sensor 18H and / or area 14.

[0066] Furthermore, the transport vehicle 256 may have a retroreflector 36M that can reflect a portion of the light 270 emitted from the light emitter 24H. The reflected light (e.g., light 276) can reach the optical sensor 18H, which can detect the light 276 and generate a third signal indicating the relative position between the transport vehicle 256 and the optical sensor 18H. The controller 16 may be configured to receive the third signal from the optical sensor 18H (e.g., via a communication component 86). In response, the controller 16 can calculate position data related to the vehicle 256. The position data may include the coordinates of the vehicle 256 (e.g., relative coordinates to the optical sensor 18H, HMDs 212 and 222, and / or area 14).

[0067] Furthermore, the controller 16 can use the coordinates of the HMD 212, HMD 222, and vehicle 256 to calculate a first relative position between the HMD 212 and vehicle 256, and a second relative position between the HMD 222 and vehicle 256. Using the first and second relative positions, the controller 16 can perform specific proactive and / or preventive actions related to the HMD 212, 222, and vehicle 256. For example, based on the first relative position indicating that the HMD 212 (and thus the guest 210) may be too close to the gate of vehicle 256, the controller 16 may determine that there is a potential maintenance problem with the HMD 212. In response to the potential maintenance problem, the controller 16 may send a command to the vehicle controller of vehicle 256 (e.g., via the communication component 86). This command may cause the vehicle controller to stop the operation of vehicle 256 for further diagnosis of the potential maintenance problem. Such proactive and / or preventative measures can improve the experience of guests 210 and 220 while they are riding in vehicle 256 within Area 14. It should be understood that similar features can be implemented for all users in Area 14 (e.g., guests and / or employees). Furthermore, the optical tracking system 12 can also be used to track other types of objects carried or worn by users (e.g., bands, shoes, removable stickers, stamps, helmets). For example, guests 210 and 220 may wear HMDs 212 and 222, which include retroreflective elements 36A and 36B that are tracked to facilitate adjustment of the image presented by the HMDs 212 and 222, and / or bands that include retroreflective elements that are tracked independently to facilitate the presentation of alerts about various maintenance issues and / or changes in operation.

[0068] Figure 6 is a flowchart of method 300 for tracking an object using the optical tracking system described herein. The optical tracking system can perform the operations described below via one or more processors (referred to as processor 82) based on processor-executable code stored in memory. The processors can execute the processor-executable code to perform object tracking based on tracking data transmitted from one or more optical sensors capable of detecting light (e.g., reflected or emitted light) from an object (e.g., object 20, HMD160, HMD212, HMD222). Based on the tracking data and other relevant data, processor 82 can calculate position data including the relative position between each object and the corresponding optical sensor 18, and / or the relative position between each object and an area (e.g., a coordinate system established for the area). Furthermore, processor 82 can transmit the respective position data to the object.

[0069] Method 300 will be described in a specific order, but Method 300 can be performed in any preferred order and is not limited to the order shown herein. Furthermore, although Method 300 below describes each processing block as being performed by an optical tracking system, other preferred computer systems can also perform the methods described herein.

[0070] Referring to Figure 6, in block 302, the optical tracking system can receive location data from an optical sensor (e.g., optical sensor 18A) that detects light from an object (e.g., object 20, HMD 160, HMD 212). The optical tracking system can receive the location data via a wired and / or wireless communication line (e.g., electrical cable 46) or wirelessly (e.g., using a wireless signal or optical signal) using a communication component. The location data includes positional information indicating the location of the object. In one embodiment, the optical sensor can generate location data based on light reflected from the object. For example, the optical sensor may include (or be coupled to) a light emitter or illuminator (e.g., light emitter 24A) that emits illumination light (e.g., infrared light or visible light). Part of the illumination light may be reflected by one or more retroreflectors (e.g., retroreflector 36A) built into or coupled to the object. The optical sensor can detect the reflected light and generate location data based on the reflected light from the object.

[0071] In one embodiment, the object may include (or be coupled to) a light-emitting element (e.g., light-emitting element 24F) capable of emitting active light (e.g., infrared or visible light). An optical sensor can detect the active light and generate location data based on the active light emitted from the light-emitting element.

[0072] In block 304, the optical tracking system can calculate positional data, including the relative position between an object and an optical sensor, based on location data. The optical tracking system can perform a series of operations for calculating positional data, such as analyzing several attributes related to light reflected or actively emitted from an object (e.g., arrival time, direction, light intensity, frequency, and / or polarization), calculating the relative distance and orientation between the object and the optical sensor, receiving or retrieving the sensor coordinates of the optical sensor (e.g., coordinates 22 including position coordinates [XC, YC, ZC] and orientation data [YAWO, PITCHO, and ROLLO]), and calculating positional data including the object coordinates of the object (e.g., coordinates 26 may include Cartesian coordinates [XO, YO, ZO] and orientation information [YAWO, PITCHO, and ROLLO]). The positional data may include relative position and orientation data between the object and the optical sensor and / or between the object and an area.

[0073] In block 306, the optical tracking system can transmit position data to an object. In one embodiment, the processor may be a component of a controller (e.g., controller 16) communicatively coupled to the optical sensor. The processor may transmit position data to the optical sensor via a wired and / or wireless communication line (e.g., electrical cable 46) or wirelessly (e.g., using a radio signal or an optical signal). The optical sensor may respond by transmitting position data to the object via an optical signal (e.g., an infrared or visible light signal) using an optical transmitter (e.g., optical transmitter 44A or emitter 24A). The object may include a photodetector capable of receiving the optical signal. The optical signal containing the position data may enable the object to recognize its position relative to the optical sensor and / or area.

[0074] In one embodiment, the processor may be a component of a device (e.g., a device coupled to or embedded in an object). The processor can transmit location data directly to the object. By recognizing the location of the object, the optical tracking system can enhance the experience of a collaborative event within an area by enabling or improving specific collaborative events (e.g., theme park events involving special effects) based on the location data.

[0075] In block 308, the optical tracking system can receive additional location data from optical sensors that detect light from further objects. For example, further objects may include another object (e.g., HMD222) that is in the vicinity of the first object (e.g., HMD212) or participating in a cooperative event with the first object. In some cases, further objects may include guests and the vehicle (e.g., vehicle 256) in which the further guests are riding.

[0076] Further objects may include (or be coupled to) one or more further retroreflectors (e.g., retroreflector 36B) built into or coupled to the further object. A portion of the illumination light emitted from the light emitter or illuminator (e.g., light emitter 24A) is reflected back to the optical sensor, which detects the reflected light and can generate further location data based on the reflected light from the further object. In one embodiment, further objects may include (or be coupled to) further light emitters that emit active light (e.g., infrared or visible light). The optical sensor can detect the active light and can generate further location data based on the active light emitted from the further light emitters of the further object.

[0077] In block 310, the optical tracking system can calculate further position data, including a first further relative position between a further object and the optical sensor, based on further location data. Calculating the first further relative position may include analyzing attributes related to light reflected or actively emitted from the further object (e.g., time of arrival, direction, light intensity, frequency, and / or polarization), calculating the relative distance and orientation between the further object and the optical sensor, extracting the sensor coordinates of the optical sensor, and calculating the first further relative position including further object coordinates and orientation information of the further object. The first further relative position may include relative position and orientation data between the further object and the optical sensor, and / or between the further object and the area.

[0078] Furthermore, in block 312, the optical tracking system can calculate further positional data, including a second further relative position between further objects, based on the relative position and the first further relative position. For example, the optical tracking system can calculate the relative position between an object and further objects based on the respective positional coordinates of the object and further objects included in the relative position and the first further relative position. The optical tracking system can also calculate the relative orientation between an object and further objects based on the azimuth coordinates of the object and further objects included in the relative position and the first further relative position.

[0079] In block 314, the optical tracking system can transmit further position data to and / or to further objects. In one embodiment, the optical tracking system can transmit position data to further optical sensors via wired and / or wireless communication lines (e.g., electrical cable 46) or wirelessly (e.g., using wireless signals or optical signals). In response, the optical sensors can transmit further position data to further objects via optical signals (e.g., infrared or visible light signals) using an optical transmitter (e.g., optical transmitter 44A or light emitter 24A). Further objects can receive the optical signals using further photodetectors. The optical signals containing the further position data can enable further objects to recognize the position of further objects relative to the optical sensors. In one embodiment, a processor may be a component of another device coupled to or incorporated into the further object. The processor can transmit further position data directly to further objects.

[0080] The optical tracking system can enhance the experience in collaborative events within an area by enabling or improving specific collaborative events (e.g., theme park events involving objects and further objects) based on location data and further location data. For example, an object can be worn by a guest (e.g., guest 210), and further objects can be worn by another guest (e.g., guest 220) who can participate in a collaborative event (e.g., a ride event using a vehicle 256). In some embodiments, the further objects may be vehicles that transport objects through the area.

[0081] In one embodiment, the optical tracking system can utilize location data and further location data to enable objects and further objects to recognize their relative positions to each other in real time. Thus, objects and further objects can enhance the experience during an event by performing or participating in the event with improved coordination. In one embodiment, the optical tracking system can generate signals (including alerts such as text messages and / or audible sounds) indicating a potential problem (e.g., a maintenance problem) based on location data and further location data. The optical tracking system can transmit signals to at least one device or system, such as a vehicle controller, an object, and / or further objects, via a wired and / or wireless communication line (e.g., an electrical cable 46) or wirelessly (e.g., using wireless or optical signals). Thus, at least one device or system can enhance the experience during an event by recognizing a potential problem and taking corresponding measures to avoid the potential problem.

[0082] Some embodiments of the optical tracking system described herein may include specific implementations of the optical tracking system. The optical tracking system may also be implemented in different ways with different components and / or different functions.

[0083] For example, in one embodiment, a specific sensor attached to or coupled to a particular tracked device (e.g., an HMD) may include a camera or angle-sensing light sensors. This embodiment can restrict the area that the tracked device "sees" to only the area that the specific camera or angle-sensing light sensor is configured to observe. As a result, the communication bandwidth can be increased when each camera or angle-sensing light sensor transmits only the relevant data to each tracked device.

[0084] In one embodiment, the flexibility of the optical tracking system can be increased by directly calculating the position data related to the tracked object on or within each optical sensor, thereby eliminating the need for an external processing unit (e.g., a controller 16).

[0085] In one embodiment, location data can be transmitted to an object along with a unique identifier and / or a timestamp (e.g., via modulation or flashing of light). In this way, the object can receive location data directed to it and / or discard data that is not timely (e.g., the timestamp indicates older data).

[0086] In one embodiment, multiple processors (e.g., processors embedded in an optical sensor) can collaboratively calculate the 3D position and orientation (e.g., position and orientation) of a tracked object. This 3D position and orientation can be transmitted to the tracked object and used to enable certain enhanced actions, such as allowing an intelligent device (e.g., an HMD with AR, other digital visual elements, sound, haptics, or holographic technology) to provide an enhanced interactive 3D version (including position and orientation) of the real-world environment. Such an interactive 3D version can further improve the use of the tracked object and the guest experience in specific individual and / or collaborative events.

[0087] The systems and methods in this disclosure provide a variety of tracking systems that can be used to track objects and provide positional data. For example, such tracking systems may include the use of data transmission via light, 2D and 3D object tracking via infrared (IR) reflection using retroreflective markers, 2D and 3D object tracking via IR light emitted from the object being tracked, "inside-out" tracking solutions that can calculate the position of the object being tracked without off-board equipment, object tracking in 2D and 3D space using lighthouse-style devices that are mounted in space to receive light signals from the object being tracked and transmit relative positional data to the object being tracked, and motion capture of virtual cameras during the shooting of visual effects for film production.

[0088] Furthermore, the systems and methods described herein enable the combination of technologies to form new solutions that improve object tracking capabilities. For example, the new solutions can utilize hardware requirements similar to those of some existing motion capture systems. Thus, the addition of further functionality related to the new solutions can be done with relatively little effort, and the new solutions can operate similarly to existing motion capture systems while mitigating problems (e.g., data transmission problems due to signal interference or network bandwidth).

[0089] While this specification illustrates and describes only some features of this embodiment, many modifications and changes will come to mind for those skilled in the art. Therefore, it should be understood that the appended claims include all such modifications and changes that constitute the precise intent of this disclosure. Furthermore, it should be understood that some elements of the disclosed embodiments can be combined or substituted with each other. It should also be understood that any feature shown in and / or described with reference to Figures 1-6 can be combined in any preferred manner. For example, the controller 16 can be included in object 20 in Figure 1 and / or object 20 in Figure 2 (for example, coordinate 26 is determined in object 20).

[0090] The claimed technologies described herein refer to and apply to tangible objects and specific examples of a practical nature that are not abstract, intangible, or purely theoretical, but which certainly improve the art. Furthermore, if any of the claims appended to the end of this specification contain one or more elements designated as "...means for performing [function]" or "...steps for performing [function]," such elements should be interpreted in accordance with 112(f) of the United States Patent Act. On the other hand, any claim containing elements designated in any other form should not be interpreted in accordance with 112(f) of the United States Patent Act. [Explanation of symbols]

[0091] 12 Optical Tracking Systems 16 Controllers 18, 18A~18D Optical Sensors 20 objects 22 Camera coordinates 24A~24D Luminous material 26 Object coordinates 30 light 36 Retroreflector 40 Reflected light 42A~42D signal 44A~44D Transmitter 46 Electrical Cables 48 light 52 axes 53 axes 56 axes 60 light 70 Reflected light 78 light 82 processors 84 memory 86 Communication Components

Claims

1. It is a system, An optical sensor configured to detect objects within an area, Controller and The controller is equipped with, The optical sensor receives first data indicating the first position of the object, The optical sensor receives second data indicating the second position, Based on the first position and the second position, calculate the position data. The position data is transmitted to the object via an optical signal. A system configured in such a way.

2. The optical sensor is configured to detect light from the object. The system according to claim 1.

3. The aforementioned light includes infrared light reflected from the object. The system according to claim 2.

4. The optical sensor includes a light-emitting element configured to emit illumination infrared light that is reflected from the object as infrared light. The system according to claim 3.

5. The light-emitting element includes one or more infrared light-emitting diodes (LEDs), one or more actively powered infrared illuminators, one or more flashing infrared light sources, one or more digital projectors or directional light sources, or any combination thereof. The system according to claim 4.

6. The object includes one or more retroreflectors configured to reflect a portion of the illumination infrared light back to the optical sensor as infrared light. The system according to claim 4.

7. The aforementioned light includes visible light reflected from the object. The system according to claim 2.

8. The light includes infrared light emitted from the light-emitting element of the object. The system according to claim 2.

9. The position data includes location data and orientation data of the object, the location data includes a two-dimensional or three-dimensional coordinate system related to the area, and the orientation data includes orientation coordinates including YAW, PITCH, and ROLL information of the object. The system according to claim 1.

10. The controller comprises one or more additional optical sensors configured to detect the object within the area, and the controller is configured to receive additional data indicating the first position of the object from the one or more additional optical sensors. The system according to claim 1.

11. The controller is configured to transmit the position data to a further object via a further optical signal. The system according to claim 1.

12. The object includes a head-mounted display configured to display an image based on the position data, The system according to claim 1.

13. The controller is integrated with the optical sensor or the object. The system according to claim 1.

14. An optical tracking system, An optical sensor configured to detect a first object and a second object within an area, Controller and The controller is equipped with, The optical sensor receives first data indicating the first position of the first object, The optical sensor receives second data indicating the second position of the second object, Receiving a third data indicating a further position of the optical sensor, Based on the first location data and the further position, the first position data of the first object is calculated. Based on the second position and the further position, the second position data of the second object is calculated. The light emitter is instructed to transmit the first position data to the first object via the first optical signal. The light emitter is instructed to transmit the second position data to the second object via a second optical signal. An optical tracking system configured as follows.

15. The first object and the second object include autonomously or remotely controlled flying objects moving within the area. The optical tracking system according to claim 14.

16. The first object and the second object each include an object light emitter configured to emit light detectable by the optical sensor, or a retroreflector configured to reflect a portion of the illumination light to generate reflected light detectable by the optical sensor. The optical tracking system according to claim 14.

17. One or more processors receive location data from an optical sensor configured to detect light from an object in an area, The process involves, via the one or more processors described above, calculating position data indicating the relative position between the object and the area based on the location data, Transmitting the position data to the object via an optical transmitter, A method that includes this.

18. The 1 or 2 or more processors receive further location data from the optical sensor configured to detect further light from further objects, The calculation of further position data indicating a first further relative position between the further object and the area, based on the further location data, via the one or more processors described above, Transmitting the further object and the further position data to the object via the optical transmitter, The method according to claim 17, including the method described in claim 17.

19. This includes projecting an image based on the position data through one or more displays associated with the object, The method according to claim 17.

20. This includes outputting one or more effects based on the position data via the output device of the object, The method according to claim 17.

Images