Calibration System and Method for Dynamic Projection Mapping

Abstract

A calibration tool for a dynamic projection mapping system includes a rigid body, a row of at least three light emitters disposed on the rigid body, and an additional light emitter disposed on the rigid body and offset from the row of at least three light emitters. The calibration tool also includes a sensor disposed on the rigid body and configured to detect projection light.

Description

Technical Field

[0001] 〔Cross - Reference to Related Applications〕 This application claims priority and the benefit thereof to U.S. Provisional Patent Application No. 63 / 350,301, entitled "CALIBRATION SYSTEMS AND METHODS FOR DYNAMIC PROJECTION MAPPING", filed on June 8, 2022, the entire disclosure of which is hereby incorporated by reference in its entirety for all purposes.

Background Art

[0002] This section is intended to introduce the reader to various aspects of the technology that may be related to the various aspects of the technology described and / or claimed hereinafter. This discussion is believed to be helpful in showing the reader the background circumstances and facilitating a better understanding of the various aspects of the present disclosure. Accordingly, these descriptions should be read from the above perspective rather than as an admission of prior art.

[0003] Amusement parks and other entertainment venues include, among other numerous attractions, animated figures that entertain guests. Among the animated figures, there are those that can be animated by projection mapping that traditionally projects a predetermined appearance onto the animated figure. For example, the animated figure can be visually complemented with a canned or fixed set of images that can cooperate with the pre-programmed movement of the animated figure. Inside the animated figure, a projector that projects an image through the translucent projection surface of the animated figure can be arranged, but the projector arranged inside may generate non-real backlighting or glow over the entire translucent projection surface of the animated figure. Currently, it is recognized that it is desirable to make the animated figure look more lifelike and give the animated figure the ability to harmonize contextually with the environment in a realistic and convincing manner.

SUMMARY OF THE INVENTION

[0004] Some embodiments within the same scope as the subject matter of the original claims are summarized below. These embodiments do not limit the scope of the present disclosure, but rather merely show an overview of possible forms of the present subject matter. In fact, the present disclosure can include various forms that may be similar to or different from the embodiments shown below.

[0005] In one embodiment, a calibration tool for a dynamic projection mapping system includes a rigid body, at least three rows of light emitters disposed on the rigid body, and an additional light emitter disposed on the rigid body and offset from the at least three rows of light emitters. The calibration tool also includes a sensor disposed on the rigid body and configured to detect projection light.

[0006] In one embodiment, a dynamic projection mapping system includes a projector configured to project visible light. The dynamic projection mapping system also includes a calibration tool having a plurality of emitters configured to emit infrared light and a sensor configured to detect the visible light projected by the projector. The dynamic projection mapping system further includes a plurality of tracking cameras configured to generate image data indicative of the infrared light emitted by the plurality of emitters. The dynamic projection mapping system further includes a processing circuit configured to establish a common origin for the projector and the plurality of tracking cameras based on sensor data received from the sensor and image data received from the plurality of tracking cameras.

[0007] In one embodiment, a method of operating a projection system and an optical tracking system for dynamic projection mapping includes instructing, via a processing circuit, a set of emitters of a calibration tool to emit light in an environment. The method also includes receiving, at a plurality of tracking cameras, image data indicative of respective positions of each emitter of the set of emitters in the environment at the processing circuit. The method further includes instructing, via the processing circuit, the projector to project visible light in the environment. The method further includes receiving, at one or more processors from a sensor of the calibration tool, sensor data indicative of the visible light detected by the sensor. The method further includes establishing, via the processing circuit, a common origin for the plurality of tracking cameras and the projector in the environment based on the image data and the sensor data.

[0008] These and other features, aspects, and advantages of the present disclosure will be better understood when the following detailed description is read with reference to the accompanying drawings in which like elements are denoted by like symbols throughout.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, one or more specific embodiments of the present disclosure will be described. For the sake of brevity in the description of these embodiments, not all implementation features may be described herein. It should be understood that in any such implementation development found in any engineering or design project, numerous implementation-specific decisions must be made to achieve the specific objectives of the developer, such as compliance with system-related and business-related constraints that may vary depending on the implementation. Further, although such development efforts can be complex and time-consuming, they should be understood as routine endeavors in design, fabrication, and manufacture for those skilled in the art who benefit from the present disclosure.

[0011] When introducing elements of various embodiments of the present disclosure, articles such as "a," "an," and "the" are intended to mean that these elements are present one or two or three or more. The terms "comprising," "including," and "having" are intended to be inclusive and mean that additional elements other than the recited elements may exist. Also, it should be understood that references to "one embodiment" or "an embodiment" of the present disclosure are not intended to be construed as precluding the existence of additional embodiments that also include the recited features.

[0012] This embodiment relates to a media system for an attraction in an entertainment environment (e.g., a theme park, amusement park, theater, stadium, concert hall). The media system can include a projector that projects an image onto the outer surface of a prop such as an animated figure. As described herein, the media system utilizes external tracking of the animated figure (e.g., via optical performance capture or optical motion capture) to dynamically generate an image and accurately project this image onto the outer surface of the animated figure.

[0013] More specifically, to assist in the accurate projection of an image onto the animated figure, a tracker can be attached to the animated figure that enables the tracking camera of the motion tracking system of the media control system to identify the motion, position, and orientation of the animated figure in real time. The media control system can operate independently of the animated figure (e.g., by not relying on position information, velocity information, and / or acceleration information from sensors or actuators of the animated figure), and can dynamically generate and adapt the projected image onto the animated figure at a realistic frame rate, such as by presenting textures, colors, and / or motions that appear indistinguishable from the animated figure, thereby mimicking a living character. As will be appreciated, the media control system can generate and update a skeletal model of the animated figure based on feedback from the tracking camera. The skeletal model generally represents the movable parts of the animated figure and is dynamically updated to represent the current three-dimensional (3D) position, orientation, and scale (e.g., the pose of the animated figure), including for example the x, y, and z coordinates, of the animated figure or a part thereof. Thus, the media control system utilizes the skeletal model to generate an image that accurately conforms to the current position and orientation of the animated figure for projection. As described herein, the calibration process can be performed to calibrate the tracking camera of the motion tracking system and the projector of the projection system with respect to a show set (e.g., a show space), and thus to calibrate the tracking camera of the motion tracking system and the projector of the projection system with respect to each other (e.g., sharing a common origin and coordinate system).

[0014] In some of the examples presented in this specification, reference is made to animated figures for ease of explanation, but this term is to be understood as having a broad scope to include any prop that can move within an attraction and / or be the subject of projection via a projection system. Generally, the techniques disclosed in this specification are to be considered applicable to projection onto any prop (e.g., an object, a structure, show action equipment [SAE]). For example, the prop can be a full animated robotic figure. As another example, the prop can be one or more objects (e.g., a building, a staff, a sword, or other object simpler than a full animated robotic figure) that move around via a complex SAE, a full animated robotic figure, and / or a participant (e.g., a human participant or an actor). In fact, the prop can be a participant (e.g., a human participant or an actor). Further, regardless of its structure, the prop can represent a character (e.g., a human-like character, an animal-like character) or can be something that does not represent a character (e.g., an inanimate object such as a building, furniture, water).

[0015] Based on these, FIG. 1 shows an attraction 10 including a prop that can be called an animation figure 12 in this specification, which receives an image 14 (e.g., projection content) from a projector 16 of the projection system of the media control system 20 (e.g., an external projector, an optical projector with a lens). As shown in the figure, the attraction 10 is a show set having a stage ceiling 22, a stage floor 24, and scenery objects 26 arranged between the stage ceiling 22 and the stage floor 24. The show set can also include any suitable stage lighting device 30, such as the lighting fixtures or lighting devices shown in the figure. From the guest area 32 of the attraction 10, a plurality of guests 34 can view the animation figure 12 and / or interact with the animation figure 12. Although the media system 8 is shown as being within a stage-type environment, it should be understood that it can be used to entertain guests 34 in any entertainment environment, such as a dark ride, an outdoor arena, and an area adjacent to the vehicle path of a vehicle carrying guests 34.

[0016] Note that the projector 16 is located outside the animation figure 12, and thus the sealed volume within the animation figure 12 can be used to accommodate components other than the projector 16, such as a specific operating system. In the illustrated embodiment, the projector 16 is arranged in front of the animation figure 12 and is blocked from the view of the guests 34 by the protrusion 36 of the stage ceiling 22. Regardless of its position, the projector 16 faces the image 14 onto the outer surface 40 of the main body 42 (e.g., structure) of the animation figure 12 that can correspond to the head 44 of the animation figure 12. Therefore, the media control system 20 can provide an immersive and interactive experience for the guests 34 by providing a realistic and attractive texture on the head 44.

[0017] The animated figure 12 is part of a motion control system 50 (e.g., a prop control system) that can operate independently of the media control system 20. For example, the motion control system 50 can utilize interaction data to dynamically update the animated figure 12. Additionally, the motion control system 50 can command the actuator to adjust the animated figure 12 and / or to adjust the position of any other suitable component of the attraction 10 visible to the guest 34. For example, the motion control system 50 can control an operable motion device 66 (e.g., an operable motion base) physically coupled to the animated figure 12. The operable motion device 66 can be any suitable motion generation assembly capable of moving (e.g., translating, rotating) the animated figure 12 in a lateral, longitudinal, and / or vertical direction. Further, it should be understood that the operable motion device 66 can also be, or include, a suspension system and / or a flight system coupled to the animated figure 12 from above the stage floor 24.

[0018] A tracker 60 (e.g., a trackable marker) can be placed on the animation figure 12. The tracker 60 can be placed on the back 62 of the animation figure 12 or on any suitable surface. The tracker 60 enables one or more tracking cameras 64 of the motion tracking system of the media control system 20 to detect or resolve the position and orientation of the animation figure 12 within the attraction 10 via optical performance capture or optical motion capture techniques, etc. Thus, as understood, the projector 16 can project the image 14 onto the animation figure 12 in synchronization with the actual current position and orientation (e.g., pose) of the animation figure 12 without relying on position information, velocity information, and / or acceleration information from sensors or actuators of the animation figure 12. However, in some embodiments, it should be understood that the media control system 20 can also verify the positioning and operation of the projector 16 based on sensor-derived information and / or actuator-derived information from the animation figure 12.

[0019] Note that it should be understood that the media system 8 can include any suitable number of projectors 16, trackers 60, and tracking cameras 64. For example, a plurality of animation figures 12 can be included within a single attraction 10, and the media system 8 can include at least one projector 16 for each animation figure 12. However, currently, the specific infrastructure of the media system 8 enables any number of animation figures 12 that can move within the light region of at least one tracking camera 64 and within the projection cone of at least one projector 16 to receive the image 14. In certain embodiments, a plurality of projectors 16 can be provided to supply content to multiple sides of a single animation figure 12. Also, some embodiments of the animation figure 12 can include at least two trackers 60 such that one or more tracking cameras 64 can resolve the relative positions of at least two trackers 60 for efficient tracking of the animation figure 12, although it should be understood that the position of a single tracker 60 can be varied to enable resolution of the position of the animation figure 12 using a less complex and / or less accurate system.

[0020] In certain embodiments, the projector 16 and the tracking camera 64 can be physically coupled to each other. For example, the projector 16 and the tracking camera 64 can be firmly attached to a frame (e.g., a rigid frame) so that they remain in fixed positions relative to each other to form an integrated system. Further, the frame can also be firmly attached to the stage floor 24 or another stationary surface of the show set. Thus, the frame can prevent (e.g., reduce or eliminate) drift between the projector 16 and the tracking camera 64, as well as drift between the projector 16 and the tracking camera 64 and the show set, during operation of the attraction 10.

[0021] Regardless of how the projector 16 and the tracking camera 64 are arranged within the attraction 10, a calibration process is executed to establish a relationship between the projector 16 and the tracking camera 64 such that the projector 16 can project the image 14 onto the animated figure 12 being tracked via the tracking camera 64. The calibration process can be performed before the operation of the attraction 10. For example, the calibration process can be performed before the start of the week, before the daily opening of the amusement park, before each cycle of the attraction 10, or any combination thereof. In certain embodiments, the calibration process is performed (e.g., triggered) in response to detected offsets between the projected image and the animated figure, etc. (e.g., the media system 8 starts recalibration when the sensor / imaging device detects an offset, and the operator visually observes the offset and provides an input to command the media system 8 to perform recalibration).

[0022] FIG. 1 also shows an example of an interaction data source 70 that includes a guest sensor 72. The guest sensor 72 can collect guest input from any of the guests 34 within the guest area 32. As recognized herein, guest input is one form of interaction data that can be utilized to adaptively update the animated figure 12 or the attraction 10. The motion control system 50 can generate a response that the animated figure 12 should execute based on the interaction data and command the actuators of the animated figure 12 to execute that response.

[0023] In one embodiment, the animation figure 12 is covered with trackers 60 (e.g., visible or invisible, active or passive retroreflective markers or active emitters). These discrete points on the animation figure 12 can be used directly as visual reference points that form the basis of a two-dimensional (2D) or three-dimensional (3D) pose estimation process. These discrete points can also be identified and supplied through a machine learning algorithm and compared against known ground truth surface poses and pose matches performed in real time.

[0024] FIG. 2 is a block diagram of a media system 8 having a media control system 20 that can operate to project an image externally onto the animation figure 12 (e.g., without communicatively coupling to the motion control system 50 or relying only on the motion control system 50). In one embodiment, the media control system 20 can be arranged not to transmit or receive direct communication signals with the motion control system 50. However, as will be described below, an interaction data source 70 can be communicatively coupled upstream of both the media control system 20 and the motion control system 50 to enable cooperation between the media control system 20 and the motion control system 50 without intercommunication between the control systems 20, 50. Directly downstream of the interaction data source 70, a network device 90, such as a switch or hub, can be communicatively coupled to facilitate efficient communication between the interaction data source 70 and the control systems 20, 50. However, it should be understood that the network device 90 can be omitted, multiple network devices 90 can be implemented, or any other suitable data management device can be utilized to facilitate the supply of data from the interaction data source 70 to the control systems 20, 50.

[0025] In the illustrated embodiment, the animated figure 12 includes a figure processor 100 and a figure memory 104, which can collectively form all or part of the figure controller 102 of the motion control system 50. On the body 42 of the animated figure 12, a tracker 60 is disposed so that the tracking camera 64 of the motion tracking system of the media control system 20 can detect the position, orientation or pose of the animated figure 12. The tracker 60 can be an active element that can each emit an individual signal to the tracking camera 64. For example, the tracker 60 can emit infrared rays, electromagnetic energy, or other suitable signals that can be detected by the tracking camera 64 (and in at least some cases, cannot be detected by the guest 34). Alternatively, the tracker 60 can be a passive element (e.g., a reflector, a pigment portion) that does not emit a signal, and the tracking camera 64 can accurately distinguish the animated figure 12 and / or other parts of the attraction 10.

[0026] Further, the animated figure 12 is attached with any suitable actuator 106 that enables the animated figure 12 to move as if it were real and alive (e.g., walking, translating, rotating, pivoting, lip synchronize). The interaction data source 70 can include any suitable data source that provides a data set that changes over time as interaction data 109. For example, the guest sensor 72 can detect the interaction between the guest and the animated figure 12 and relay the interaction data indicating the guest's interaction to the figure controller 102. Thereafter, the figure controller 102 can command the actuator 106 to dynamically operate the animated figure 12 to immediately respond to the interaction data 109.

[0027] The media control system 20 can include a projector 16, a tracking camera 64, a camera network device 110, and / or a media controller 112. The media controller 112 can be communicatively coupled to an interaction data source 70 (e.g., via a network device 90) to dynamically respond to interaction data 109 and / or other changes in the attraction 10. In some embodiments, the media control system 20 can be communicatively separated from the motion control system 50. That is, the motion control system 50 can be independent of the media control system 20. Thus, the media control system 20 provides the animation figure 12 with the freedom of operation to adaptively respond to the interaction data 109 substantially in real time (e.g., within microseconds or milliseconds of the interaction), while monitoring or tracking the motion of the animation figure 12 and also projecting an image onto the animation figure 12 substantially in real time. Accordingly, the media control system 20 simultaneously executes a media feedback loop that modifies the image projected onto the animation figure 12 while the motion control system 50 is executing a figure feedback loop.

[0028] The media control system 20 utilizes the tracking camera 64 to collect information regarding the current position and orientation of the animation figure 12. The type or configuration of the tracking camera 64 can be individually selected to correspond to the type of the tracker 60 and also to detect the type of the tracker 60. The positioning of these trackers 60 is coordinated with a geometric model or a skeletal model of the animation figure 12 to facilitate adjustment of the projection onto the animation figure 12 in different orientations.

[0029] The tracking camera 64 is communicably coupled to a camera network device 110 that relays to the media controller 112 a signal indicative of the current three-dimensional position and orientation (e.g., the pose of the animated figure 12) of the animated figure 12 or a portion thereof (e.g., including the x, y, and z coordinates relative to the origin). Thus, the camera network device 110 is a network switch or sensor hub that integrates multiple information streams from the tracking camera 64 for efficient processing by the media controller 112. The media controller 112 includes a media processor 114 and a media memory 116 that cooperate to determine, generate, and / or adjust a moving image to be overlaid on the animated figure 12 at the current position and orientation of the animated figure 12. As a result, the media controller 112 can instruct the projector 16 to project the moving image onto the animated figure 12. The image can be rendered on demand in its entirety based on the current pose (e.g., position and orientation) of the animated figure 12. In a low-complexity configuration, the image can be generated by adapting a pre-recorded video stream to the current pose of the animated figure 12. The media controller 112 can be any suitable media generator or game engine with high processing power and low latency. Thus, it should be understood that the media controller 112 can generate the image projected onto the animated figure 12 substantially in real time based on the data received from the tracking camera 64. In practice, the media controller 112 can maintain a skeletal model or algorithm representing the animated figure 12 and its operable parts (e.g., jaws, hands, feet, joints). The media controller 112 can update the skeletal model to represent the actual current position and orientation of the animated figure 12 based on the data and then generate an image to be projected onto the animated figure 12 having this current position and orientation.

[0030] The projector 16 can include a projector processor 120 and a projector memory 122 that facilitate the presentation of an image onto the animation figure 12. The projector processor 120 generally receives data representing an image from the media controller 112 and then instructs the light source within the projector 16 to output the image through a lens. The media controller 112 can determine the current silhouette of the animation figure 12 or the shape of the target figure portion that is scheduled to receive the projected image based on the updated skeletal model and then instruct the projector 16 to provide the image on the silhouette.

[0031] Each of the processors 100, 114, 120 can be any suitable processor capable of executing instructions for implementing the techniques of the present disclosure, such as a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), a processor of a programmable logic controller (PLC), a processor of an industrial personal computer (IPC), or some other similar processor configuration. These instructions are encoded in a program or code stored in a tangible, non-transitory computer-readable medium such as the memories 104, 116, 122 and / or other storage circuitry or storage devices. Accordingly, the figure processor 100 is coupled to the figure memory 104, the media processor 114 is coupled to the media memory 116, and the projector processor 120 is coupled to the projector memory 122. This embodiment of the media system 8 also includes a show control system 130 that coordinates additional output devices of the attraction 10. For example, the show controller 132 of the show control system 130 is communicatively coupled between the network device 90 and one or more lighting output devices 134, audio output devices 136, and / or venue-specific special effects output devices 138 (e.g., a fog machine, a vibration generator, an operable portion of the scenic object 26).

[0032] FIG. 3 is a front view of an image 14 provided on the head 44 of the main body 42 of the animated figure 12. The image 14 can include features or textures similar to a face. For example, eyebrows, eyes, a nose, lips, and / or wrinkles can be projected onto the head 44. The animated figure 12 is wearing clothing elements (e.g., a hat, a wig, jewels), and the media controller 112 and / or the projector 16 can identify the outline of the outer surface 40 of the animated figure 12 formed by the clothing elements (e.g., via projection masking). As a result, the projector 16 directs the image 14 to a target portion or figure portion of the outer surface 40 of the animated figure 12. The media control system 20 monitors the movement of the animated figure 12, such as large movements across the stage and / or small movements of the articulating jaw, to project an appropriate and realistic image onto the head 44 of the animated figure 12.

[0033] FIG. 4 is a front view of an image 14 provided on the outer surface 40 of the animated figure 12. As shown, the image 14 provides the animated figure 12 with a character, non-human appearance, or fantastical appearance, such as the appearance of an owl. The outer surface 40 of the head 44 can be textured to complement the image 14. Also, it should be understood that the image 14 can include supernatural, fantastical, or non-human images and / or effects, such as flames, smoke, transformation, and color morphing.

[0034] By referring to FIGS. 5-8, aspects related to the calibration and alignment of the projector 16 and the tracking camera 64 can be better understood. FIG. 5 shows the attraction 10 including the tracking camera 64 and the projector 16. A calibration process can be performed using a calibration tool 150 (e.g., a calibration wand or device) within the show set. As part of an initial portion of the calibration process (e.g., camera calibration to calibrate the tracking cameras 64 to each other), an operator 152 (e.g., a human operator, an autonomously or remotely controlled robot) can carry the calibration tool 150 within the show set. The operator 152 can move across the show set while moving the calibration tool 150 around the show set (e.g., walking or advancing from one side of the show set to another side of the show set) by, for example, waving the calibration tool 150 back and forth. The calibration tool 150 includes a plurality of emitters 154, such as at least three emitters 154, to facilitate the initial portion of the calibration process. The plurality of emitters 154 can be arranged in a single line or in a row at known relative positions on the calibration tool 150. In one embodiment, the plurality of emitters 154 can be light emitters (e.g., light emitting diodes [LEDs]). For example, the plurality of emitters 154 can be light emitters that emit infrared (IR) light that is detectable by the tracking camera 64 (and not visible or detectable by the guest 34).

[0035] As the calibration tool 150 progresses within the show set, each of the tracking cameras 64 captures a plurality of (e.g., dozens, hundreds, thousands of) image frames. The media controller 112, or other suitable processing circuitry of the media system 8, can process the plurality of image frames to calibrate the tracking cameras 64 with respect to each other. For example, the media controller 112, or other suitable processing circuitry of the media system 8, can compare the plurality of frames with respect to each other to determine the relative positions of the tracking cameras 64 (e.g., after comparing a plurality of frames from different tracking cameras 64 at a first position with the calibration tool 150, comparing a plurality of frames from different tracking cameras 64 at a second position with the calibration tool 150, and so on). The initial portion of the calibration process conducted in this manner can also advantageously account for / compensate for variations in the characteristics of the tracking cameras 64, such as lens distortion of the tracking cameras 64.

[0036] As shown, the calibration tool 150 also includes at least one additional emitter 156 (e.g., an offset emitter) and at least one sensor 158 (e.g., a photodetector). The calibration tool 150 can be used to perform the calibration process periodically (e.g., before the start of the week, daily before the amusement park opens, before each cycle of the attraction 10, or any combination thereof) and / or at other suitable times (e.g., in response to a particular event). Further, the calibration tool 150 can be used only within the show set during the calibration process and is not used during the presentation of the show within the show set. However, in one embodiment, the calibration tool 150 can include or incorporate an object of interest (e.g., a prop) onto which the projector 16 projects an image during the show. For example, the calibration tool 150 can be part of a show action equipment (SAE) that is projected during the show and / or appears only during a portion (e.g., the start portion; the initial portion) of the show.

[0037] FIG. 6 shows an attraction 10 including a tracking camera 64 and a projector 16 arranged in a show set so that a calibration tool 150 establishes an origin (e.g., a common origin). The origin can be used to calibrate the tracking camera 64 and the projector 16 with respect to the show set and can also effectively calibrate the tracking camera 64 and the projector 16 with respect to each other.

[0038] The calibration tool 150 can be placed and held at an origin setting position in the show set as part of an additional part of the calibration process (e.g., origin calibration for establishing an origin in the show set). For example, the calibration tool 150 can be fastened (e.g., bolted) and / or otherwise fixed (e.g., via an interference fit) to a structure 160 (e.g., a stationary structure) in the show set. In one embodiment, the structure 160 can be attached (e.g., bolted) to the stage floor 24 or other surface in the show set so that the structure 160 remains stationary with respect to the show set. In one embodiment, the structure 160 can include a bracket 162 configured to support a part of the calibration tool 150. For example, in FIG. 6, the structure 160 includes a bracket 162 having a recess configured to receive a part of the calibration tool 150. In this way, the calibration tool 150 can be firmly coupled to the structure 160 and held at the origin setting position in the show set. However, it should be understood that the calibration tool 150 can be placed and held at the origin setting position via any suitable technique.

[0039] During an additional part of the calibration process, at least three emitters on the calibration tool 150 are visible to the tracking camera 64. In one embodiment, the at least three emitters can include two of the emitters 154 arranged in a triangle (such as three points that form or define a triangular area or region across the calibration tool 150, for example) and an additional emitter 156. Since the tracking cameras 64 are already calibrated to each other in an initial part of the calibration process, at least three emitters arranged in a triangle can be tracked in three dimensions. The tracking camera 64 can capture an image frame, and the media controller 112, or any other suitable processing circuitry of the media system 8, uses the respective positions of at least three emitters (such as two emitters 154 and an additional emitter 156) within the image frame to set an origin for the tracking camera 64 within the show set. For example, the origin can be set to coincide with the respective centers of the additional emitters 156, the respective centers of the sensors 158, or any other suitable position. Further, since the at least three emitters have known relative positions and / or spacing on the calibration tool 150, the media controller 112, or any other suitable processing circuitry of the media system 8, uses the respective positions of at least three emitters (such as two emitters 154 and an additional emitter 156) within the image frame to set a coordinate system based on the origin for the tracking camera 64 within the show set. In this way, the media controller 112, or any other suitable processing circuitry of the media system, can set an origin (0, 0, 0) and a coordinate system having an x-axis or x-direction, a y-axis or y-direction, and a z-axis or z-direction that is implied as protruding perpendicularly from a 90-degree corner of the triangle.

[0040] Next, as part of the sensor mode portion of the calibration process, command a plurality of emitters 154, 156 (e.g., all emitters) to light up, and move the calibration tool 150 around the show set while commanding the projector 16 to emit light (e.g., structured light scanning) towards the calibration tool 150 within the show set. In one embodiment, to execute the sensor mode portion of the calibration process, move the calibration tool 150 to additional positions around the show set (e.g., sequentially execute these steps with the calibration tool 150 existing at different positions other than the origin set position). However, it is also envisioned that an additional portion of the calibration process and a portion of the sensor mode portion of the calibration process can be executed simultaneously (e.g., execute some steps simultaneously or at overlapping times at the origin set position, and then move the calibration tool 150 to an additional position to complete the sensor mode portion of the calibration process). In any case, during the sensor mode portion of the calibration process, the calibration tool 150 is placed at a plurality of positions such as at least 6 locations. For ease of explanation, FIG. 6 includes a plurality of additional structures 164 (e.g., at least 6 additional structures 164 arranged at at least 6 different positions around the show set) at different positions around the show set. The additional structures 164, if present, can include any of the features of the structure 160 (e.g., fastened to the show set with respective brackets). During operation, when the additional portion of the calibration portion is completed and the origin is set, the calibration tool 150 can be moved sequentially from the structure 160 to each of the additional structures 164 (e.g., coupled sequentially to each of the additional structures 164, such as to the second additional structure among the additional structures 164 after the first additional structure among the additional structures 164). However, it should be understood that the disclosed technique can also be executed without using the additional structures 164, and instead the operator can carry the calibration tool 150 or arrange it in other forms at different positions around the show set.

[0041] In any case, at each of a plurality of positions in the sensor mode portion of the calibration process, the media controller 112, or any other suitable processing circuit of the media system 8, commands a plurality of emitters 154, 156 to light up so as to be able to track the calibration tool 150 (for example, the body of the calibration tool 150). Also, simultaneously, at each of a plurality of positions in the sensor mode portion of the calibration process, the projector 16 is commanded to emit light (for example, structured light scanning) towards the calibration tool 150 in the show set. The sensor 158 detects the light emitted by the projector 16 and provides data (for example, sensor data, signals) to the media controller 112 or any other suitable processing circuit of the media system 8. This data is processed to determine which (single or multiple) pixels of the light from the projector 16 reached the sensor 158 (for example, the x, y pixel positions of that position of the sensor 158). These techniques result in the real-world offset position of the sensor 158 (for example, the origin and the relative position with respect to the coordinate system established for the tracking camera 64). Therefore, this can be used to set the origin and the coordinate system in the show set for the projector 16 (for example, to determine the pose of the projector 16 with respect to the origin and the coordinate system). In this way, both the tracking camera 64 and the projector 16 are calibrated with respect to the show set (for example, based on a common origin and coordinate system).

[0042] The origin can establish a coordinate system (e.g., two - dimensional or three - dimensional, the relative coordinate system of the attraction 10) that does not change during the cycle of the attraction 10. As a result, the tracking camera 64 tracks the animation figure 12 within the coordinate system with reference to the origin and the coordinate system. Also, as shown in FIG. 6, the projector 16 can also project the image 14 accurately onto the animation figure 12 (e.g., constantly, in all poses) during the cycle of the attraction 10 with reference to the origin and the coordinate system. In this way, the tracking camera 64 and the projector 16 are calibrated and aligned with each other. During the operation in the cycle of the attraction 10, when the tracking camera 64 detects that the animation figure 12 (e.g., a specific vertex) is in the first coordinate set, the media controller can instruct the projector 16 to project an image onto the animation figure 12 in the first coordinate set. Since the tracking camera 64 and the projector 16 are calibrated and aligned with each other, the image is correctly aligned and mapped onto the animation figure 12.

[0043] Note that the media controller 112 can generally operate as a two-dimensional solution (e.g., within an XY coordinate system) such that the animation figure 12 is captured using the tracking camera 64, the features or markers of the animation figure 12 are identified in a two-dimensional space using the common X / Y origin of the tracking camera 64 and the projector 16, and the image is directly mapped onto the animation figure 12 in the two-dimensional space. In certain embodiments, the media controller 112 can generally operate as a three-dimensional solution (e.g., within an XYZ coordinate system). In such cases, machine learning can be used to solve for the pose estimation of the animation figure 12 in three-dimensional space. If the animation figure 12 has a face, this machine learning can be a type of face tracking in which a machine learning model is trained based on a large set of labeled and tagged face images that generally show poses, expressions, symmetry, and surface features. The resulting pose estimation can then be used to project masks or digital costumes and effect elements in real time.

[0044] FIG. 7 is a front view of an embodiment of a calibration tool 150 that can be used as part of a media system 8. As shown, the calibration tool 150 includes a body 170 (e.g., a rigid body). The body 170 can include a handle portion 172 configured to be grasped by an operator 152. The body 170 includes or can be coupled to a cable 174 that conveys commands and data between the calibration tool 150 and other devices and / or systems such as a media controller 112. For example, the cable 174 can convey commands from the media controller 112 that cause certain emitters 154, 156 (e.g., all emitters 154, two of the emitters 154, and additional emitters 156) to light, and / or convey data from a sensor 158. It should also be understood that the cable 174 can supply power to the calibration tool 150. Further, in one embodiment, the calibration tool 150 can include a communication device configured to communicate via a wireless protocol (e.g., Wi-Fi, Bluetooth) and / or an internal power source (e.g., a battery such as a rechargeable and / or replaceable battery). In some such cases, the calibration tool 150 may not include the cable 174 (or any cable) or may not be coupled to the cable 174. Further, for ease of explanation, the cable 174 is shown extending from the handle portion 172, but it should be understood that the cable 174 can extend from any suitable portion of the calibration tool 150 (e.g., a portion other than the handle portion 172 or the end portion such that the handle portion 172 or the end portion can be easily fitted into the bracket 162 or otherwise coupled to the structure 160).

[0045] As shown in the figure, the main body 170 includes a cross-shaped design having a first arm 176 (e.g., a horizontal arm) and a second arm 178 (e.g., a vertical arm) that crosses (e.g., is orthogonal to) the first arm. The first arm 176 and the second arm 178 can be independent structures fastened (e.g., bolted) to each other, or the first arm 176 and the second arm 178 can be integrally formed (e.g., integrally molded). The emitters 154 are distributed in a single line or in a row on the calibration tool 150 and can be distributed, for example, in a single line or in a row across the first arm 176. The emitters 154 can be present at known relative positions (e.g., at known intervals) on the calibration tool 150 and may not be equally spaced from each other. For example, the first distance 180 between the first emitter 154 and the second emitter 154 can be different from the second distance 182 between the second emitter 154 and the third emitter 154. This different spacing can facilitate the calibration process by enabling the detection of the orientation of the calibration tool 150 within the show set based on the different spacings reflected in the plurality of image frames captured by the tracking camera 64. The emitters 154 can form a first group of emitters 184 (e.g., a wand emitter, a camera alignment emitter).

[0046] As shown in the figure, the calibration tool 150 also includes an additional emitter 156. The additional emitter 156 can be used together with any two of the emitters 154 to establish the origin of the tracking camera 64. The additional emitter 156 and the two emitters 154 can be arranged to form a triangle (for example, a right triangle where each emitter forms a point of the triangle rather than an equilateral triangle). The additional emitter 156 and the emitters 154 can be present at known relative positions (e.g., at known intervals) on the calibration tool 150 and may not be equidistant from each other. For example, the first distance 180 between the first emitter 154 and the second emitter 154 can be different from the third distance 186 between the additional emitter 156 and the second emitter 154. This different spacing can facilitate the calibration process by enabling the detection of the orientation of the calibration tool 150 within the show set based on the different spacings reflected in the plurality of image frames captured by the tracking camera 64. The additional emitter 156 and the two emitters 154 can form a second group of emitters 188 (e.g., camera origin emitters).

[0047] As shown in the figure, the calibration tool 150 also includes a sensor 158. The sensor 158 can represent one or more sensors. For example, the sensor 158 can represent or include a plurality of sensors that detect light corresponding to a plurality of pixels of the light from the projector 16. As another example, the sensor 158 can represent or include one sensor that detects light corresponding to one pixel (e.g., a narrow sensor area) of the light from the projector 16 or a plurality of pixels (e.g., a wide sensor area) of the light from the projector 16. As yet another example, the sensor 158 can represent or include one sensor that detects the aligned / corresponding (single / multiple) pixels of the light from the projector 16 over a plurality of structured light scans from the projector 16. In any case, the sensor 158 can be used to establish the origin of the projector 16.

[0048] In FIG. 7, emitter 154 and additional emitter 156 are disposed on the front side (e.g., surface) of calibration tool 150, and sensor 158 is disposed on the back side (e.g., back surface) opposite to the front side of calibration tool 150. This arrangement can facilitate the completion of the calibration process in a show set including a tracking camera 64 on one side and a projector 16 on another side (e.g., the opposite side) as shown in FIG. 1. Generally, it is considered desirable to dispose the tracking camera 64 behind the animation figure 12 such that the tracker 60 is disposed on the back surface of the animation figure 12 (e.g., not visible to a guest looking at the front of the animation figure 12), and to dispose the projector 16 in front of the animation figure 12 to project light onto the front surface (e.g., visible to the guest) of the animation figure 12. However, in order to facilitate the completion of the calibration process in a show set including the tracking camera 64 and the projector 16 on one side (e.g., the same side or surface), the emitter 154, the additional emitter 156, and the sensor 158 can also be disposed on one side (e.g., the same side) of the calibration tool 150. In practice, the emitter 154, the additional emitter 156, and the sensor 158 can be disposed on any side of the calibration tool 150 (e.g., the same side or surface, the opposite side or back surface, and / or different sides or surfaces including any front side / side / back side or front surface / side surface / back surface) in order to facilitate the completion of the calibration process in a show set including the tracking camera 64 and the projector 16 at various positions in the show set.

[0049] The disclosed technique provides a real-world offset position of the sensor 158. When the sensor 158 detects light emitted by the projector 16, the data output by the sensor 158 indicates the (single or plural) pixels aligned with the sensor 158. Accordingly, the media controller 112, or other suitable processing circuitry of the media system 8, can align the projector 16 to the origin (e.g., establish a common origin of the tracking camera 64 and the projector 16).

[0050] Note that it should be understood that the calibration tool 150 is assumed to include variations including variations in the number and / or arrangement of the emitter 154, the additional emitter 156, and the sensor 158. For example, the emitter 154 can be arranged linearly or in a row along the second arm 178. Also, as another example, the additional emitter 156 can be used with a greater number of additional emitters 156 (e.g., instead of being used with the emitter 154 which is also used to calibrate the tracking cameras 64 with each other) (e.g., the emitter 154 cannot be reused in an additional part of the calibration process). Specifically, the calibration tool 150 can include three or more emitters 154 in the first emitter group 184 to calibrate the tracking cameras 64 with each other, and three or more additional emitters 156 in the second emitter group 188 to establish the origin of the tracking cameras 64 (e.g., including at least a total of six emitters 154, 156 instead of at least a total of three emitters 154, 156 shown in FIG. 7). Further, the sensor 158 can be arranged at the same position as one of the emitters 154, 156. For example, the sensor 158 can be arranged on the back side of the calibration tool 150, directly behind or in front of the additional emitter 156 on the front side of the calibration tool 150. Also, as described above, the emitter 154, the additional emitter 156, and the sensor 158 can be arranged on one side (e.g., the same side or surface) of the calibration tool 150.

[0051] The active light emitters 154, 156 can facilitate detection by the tracking camera 64. However, it should be understood that one or more of the emitters 154, 156 can be replaced with passive devices that do not emit light or signals detectable by the tracking camera 64. For example, retroreflective markers can be placed at the locations of the illustrated emitters 154, 156 such that the retroreflective markers can reflect light detectable by the tracking camera 64. In such cases, the calibration tool 150 can include a movable cover (e.g., movable via an electronically controlled actuator and / or manually adjustable) that covers the retroreflective markers that are not desired or not used during the current portion of the calibration process. For example, during an initial portion of the calibration process, the retroreflective marker placed at the location of the additional emitter 156 shown in the figure is covered, and during an additional portion of the calibration process, at least one of the retroreflective markers placed at the location of one of the emitters 154 shown in the figure is covered.

[0052] The sensor 158 can be a visible light sensor (e.g., a photodiode) capable of detecting light from the projector 16 (e.g., the light from the projector 16 can be only light within the visible light spectrum). Further, the emitters 154, 156 can be infrared (IR) light emitting diodes (LEDs) such that the tracking camera 64 can easily detect the light from the emitters 154, 156 (e.g., the tracking camera can capture only light of wavelengths associated with IR light). However, the sensor 158 can detect any type of light (e.g., a first type of light), and the emitters 154, 156 can emit any type of light (e.g., a second type of light that is the same as or different from the first type of light).

[0053] Emitters 154, 156 and sensor 158 provide different functionalities. For example, the purpose of emitters 154, 156 is to provide tracking points or “markers” for tracking camera 64. Any of various IR LEDs can be utilized as emitters 154, 156, and the light output (beam angle) of the emitter is equivalent to the specifications of the IR LED. In one embodiment, emitters 154, 156 can emit light having a wavelength of about 850 nanometers (nm). Sensor 158 is used to detect visible light from projector 16. The diameter of sensor 158 (e.g., about 1 millimeter [mm]) can have a size corresponding to the size of one pixel at the target pixel pitch (e.g., 0.05 inches or 1.27 mm per pixel), although sensor 158 can also be larger or smaller than one pixel. In another embodiment, the diameter of sensor 158 can be about 0.5 mm. Sensor 158 can have a peak response in the light spectrum visible to humans. Sensor 158 is a high-resolution (16-bit) ambient light sensor and preferably provides a linear response in the range of 0 to 65k lux. In one embodiment, sensor 158 can enable sub-pixel (e.g., projector pixel) accuracy by having a read value that increases as light approaches the center of sensor 158. In one embodiment, sensor 158 can be a small sensor array (e.g., a phototransistor array) that achieves a similar result. In one embodiment, sensor 158 is not affected by IR light (including light leakage from emitters 154, 156). In one embodiment, sensor 158 is not a photore resistor or a phototransistor.

[0054] In one embodiment, emitters 154, 156 are always on (lit). Alternatively, emitters 154, 156 can also be controllable via a simple negative-positive-negative (NPN) digital I / O bit or the like. In one embodiment, sensor 158 is configured to convert visible light into an analog signal that is directly output as an analog output (e.g., 0-5V, 0-10V, 0-15V, 0-20V, 5-10V, 5-15V, 5-20V) or detected as a threshold on a sensor amplifier. When an adjustable threshold is detected, an NPN digital output is triggered. The bandwidth or scanning frequency of the sensor is at least 50 Hz and can ideally be at least 100 Hz (or at least 150 Hz, 200 Hz, 250 Hz). The compatible voltage of the system can be 24Vdc, 5Vdc, or any other suitable Vdc. Emitters 154, 156 and sensor 158 can also be supported on a printed circuit board (PCB) to facilitate, for example, the adjusted light emission by emitters 154, 156 and the processing and communication of the light detected via sensor 158. The PCB can also provide a rigid substrate that maintains the fixed relative positions of emitters 154, 156 and sensor 158.

[0055] Figure 8 is a flowchart of an embodiment of a method 200 for operating the media system 8 of FIG. 1. Method 200 includes various steps represented by blocks. Note that method 200 can be executed as an automated procedure by a system such as the media system 8 of FIG. 1. Although the flowchart shows the steps in a particular order, it should be understood that these steps can be executed in a suitable order and, where appropriate, several steps can be executed simultaneously. Further, some steps or portions of method 200 can be executed by another system or device.

[0056] Method 200 can start by moving a calibration tool within an environment (e.g., an attraction show set) in block 202 to calibrate a plurality of tracking cameras with each other. As part of an initial portion of this calibration process, an operator (e.g., a human operator, an autonomous or remotely controlled robot) can carry the calibration tool within the environment. The operator can also move across the environment while moving the calibration tool around the environment, such as by swinging the calibration tool back and forth. The calibration tool includes a plurality of emitters, such as at least three emitters, to facilitate an initial portion of the calibration process. The plurality of emitters can be arranged in a single line or in a row at known relative positions on the calibration tool. In one embodiment, the plurality of emitters can be light emitters (e.g., light emitting diodes [LEDs]). For example, the plurality of emitters can be light emitters that emit infrared (IR) light that can be detected by the tracking cameras (and that is not visible or detectable by guest 34).

[0057] As the calibration tool moves within the environment, each of the tracking cameras captures a plurality of (e.g., dozens, hundreds, thousands of) image frames. A media controller or other suitable processing circuit can process the plurality of image frames to calibrate the tracking cameras with each other. For example, the media controller or other suitable processing circuit can compare the plurality of frames with each other to determine the relative positions of the tracking cameras (e.g., after comparing a plurality of frames from different tracking cameras at a first position with the calibration tool, comparing a plurality of frames from different tracking cameras at a second position with the calibration tool, and so on).

[0058] In block 204, method 200 can proceed to set a calibration tool at a location within the environment (e.g., a home position). The calibration tool can be fastened (e.g., bolted) and / or otherwise secured (e.g., via an interference fit) to a structure (e.g., a stationary structure) within the environment as part of this additional portion of the calibration process. In some embodiments, the structure can be attached (e.g., bolted) to a stage floor or other surface within the environment so that the structure remains stationary relative to the environment. In one embodiment, the structure can include a bracket configured to support a portion of the calibration tool. In this way, the calibration tool can be firmly coupled to the structure and held in position within the environment. However, it should be understood that the calibration tool can be positioned and held at the home position via any suitable technique.

[0059] In block 206, method 200 can proceed to detect or set the origin using a tracking camera. During this additional portion of the calibration process, at least three emitters on the calibration tool are visible to the tracking camera. In one embodiment, the at least three emitters can include two of the emitters used in block 202 and an additional emitter arranged in a triangle (e.g., as three points forming a triangle across the calibration tool). The tracking camera can capture an image frame, and a media controller or any other suitable processing circuit can use the respective positions of the at least three emitters within the image frame to set an origin for the tracking camera within the environment. For example, the origin can be set to coincide with the respective centers of the additional emitters, the respective centers of the sensors, or any other suitable position. Further, since the at least three emitters have known relative positions and / or spacing on the calibration tool, the media controller or any other suitable processing circuit can use the respective positions of the at least three emitters within the image frame to set up a coordinate system based on the origin for the tracking camera within the environment.

[0060] In block 208, method 200 can proceed to move calibration tool 150 to multiple additional positions within the show set (e.g., at least 6 additional positions within the show set). Further, block 208 of method 200, at each of the multiple additional positions within the show set, includes emitting light from at least the emitters (e.g., multiple emitters, all emitters on the calibration tool) and detecting the light output by the projector via the sensors of the calibration tool (e.g., structured light scanning) to calibrate the projector (e.g., determine the pose of the projector relative to the origin and coordinate system and establish a common origin and coordinate system for the tracking camera / motion tracking system and the projection / projector system). As part of the sensor mode portion of this calibration process, the projector is instructed to emit light (e.g., structured light scan) towards the calibration tool within the environment. Blocks 202, 204, 206, and 208 can be executed cooperatively via electronic control signals from the media controller (e.g., automatically) and / or via manual input from a human operator (e.g., manually).

[0061] In either case, the sensor detects the light emitted by the projector and provides data (e.g., sensor data, signals) to the media controller or any other suitable processing circuit. This data is processed to determine which pixels of the light from the projector reached the sensor. These techniques provide the real-world offset position of the sensor. Thus, this can be used to set the origin and coordinate system for the projector within the environment (e.g., determine the pose of the projector relative to the origin and coordinate system). In this way, both the tracking camera and the projector are calibrated with respect to the environment (e.g., based on a common origin and coordinate system). Specifically, an algorithm within the media controller equalizes (e.g., associates) the tracking camera and the projector such that the coordinates (X, Y, Z) of the origin within the space of the tracking camera are equal to the pixel positions (X1, Y1) relative to the raster of the projector.

[0062] The origin and the coordinate system do not change during the attraction cycle. Thus, the tracking camera tracks the animation figure within the coordinate system with reference to the origin and the coordinate system. Also, the projector can accurately project an image onto the animation figure during the attraction cycle (e.g., at all times and in all poses) with reference to the origin and the coordinate system. In this way, when the tracking camera detects that the animation figure (e.g., a specific vertex) exists in the first coordinate set during the attraction cycle, the media controller can instruct the projector to project an image onto the animation figure at the first coordinate set. Since the tracking camera and the projector are calibrated and aligned with each other, the image is correctly aligned and mapped onto the animation figure.

[0063] The calibration tool disclosed herein advantageously includes both a wand for calibrating the tracking camera (e.g., via a single linear emitter) and a triangle emitter that operates as both a triangle for calibrating the tracking camera (e.g., via three emitters forming the points of a triangle), as well as projector calibration of the projector (e.g., via a sensor). It should be understood that any features illustrated or described with reference to FIGS. 1 - 8 can be combined in any suitable form. Although only some features of the present disclosure have been illustrated and described herein, many modifications and variations will occur to those skilled in the art. Accordingly, it should be understood that the appended claims are intended to cover all such modifications and variations that fall within the true spirit of the present disclosure.

[0064] The technology claimed in this specification is applicable to tangible things and specific examples of a practical nature that reliably improve this technical field, and thus is not an abstract, intangible, or purely theoretical thing. Further, if any claim appended at the end of this specification includes one or more elements designated as "means for [performing]... [function]" or "steps for [performing]... [function]", such elements shall be construed in accordance with 35 U.S.C. § 112(f). On the other hand, for any claim that includes elements designated in any other form, such elements shall not be construed in accordance with 35 U.S.C. § 112(f).

Explanation of Signs

[0065] 10 Attraction 16 Projector 64 Tracking Camera 150 Calibration Tool 152 Operator 154 Emitter 156 Additional Emitter 158 Sensor

Claims

1. A calibration tool for dynamic projection mapping systems, A rigid body including a first side and a second side opposite to the first side, A row of at least three light-emitting elements arranged on the first side of the rigid body, An additional light-emitting element is positioned on the first side of the rigid body and offset from the row of at least three light-emitting elements, A sensor is positioned on the second side of the rigid body and configured to detect projected light, A proofreading tool equipped with the following features.

2. The calibration tool according to claim 1, wherein two of the light-emitting elements in the row of at least three light-emitting elements and the additional light-emitting element are configured to form a triangle of light-emitting elements.

3. The calibration tool according to claim 1, wherein the at least three light-emitting elements and the additional light-emitting elements are configured to emit infrared light.

4. The calibration tool according to claim 1, wherein the sensor is configured to detect visible projected light.

5. The calibration tool according to claim 1, wherein the rigid body includes a handle portion configured to be gripped by an operator.

6. The calibration tool according to claim 1, wherein the rigid body includes a first arm and a second arm that crosses the first arm.

7. A dynamic projection mapping system, A projector configured to project visible light, Multiple emitters configured to emit infrared light, and A sensor configured to detect the visible light projected by the projector, Includes proofreading tools, Multiple tracking cameras configured to generate image data showing the infrared light emitted by the multiple emitters, A processing circuit configured to establish a common origin for the projector and the multiple tracking cameras based on the sensor data received from the sensor and the image data received from the multiple tracking cameras, A dynamic projection mapping system equipped with [features / equipment].

8. The dynamic projection mapping system according to claim 7, wherein the calibration tool includes a rigid body having a first side and a second side opposite to the first side, the plurality of emitters are arranged on the first side and the sensors are arranged on the second side.

9. The dynamic projection mapping system according to claim 7, wherein the plurality of emitters include a row of at least three emitters and additional emitters offset from the row of at least three emitters.

10. The dynamic projection mapping system according to claim 9, wherein two emitters in the row of at least three emitters and the additional emitter are arranged to form an emitter triangle.

11. The dynamic projection mapping system according to claim 10, wherein the processing circuit is configured to command the row of at least three emitters to emit the infrared light, and while the row of at least three emitters is emitting the infrared light, the plurality of tracking cameras are calibrated against each other based on additional image data generated by the plurality of tracking cameras, and thereafter the processing circuit is configured to command the triangle of emitters to emit the infrared light, and while the triangle of emitters is emitting the infrared light, the common origin for the plurality of tracking cameras is established based on the image data generated by the plurality of tracking cameras.

12. The dynamic projection mapping system according to claim 7, wherein the calibration tool includes a handle portion configured to be grasped by an operator.

13. The dynamic projection mapping system according to claim 7, wherein the plurality of tracking cameras are configured to track the prop based on the detection of one or more trackers coupled to the prop, and the projector is configured to project an image onto the prop as the prop moves through the environment.

14. A prop configured to move within the environment, One or more trackers on the aforementioned prop, Equipped with, The plurality of tracking cameras are configured to generate additional image data representing the one or more trackers in the environment based on the detection of the one or more trackers on the prop by the plurality of tracking cameras. The processing circuit is configured to determine the position of the prop relative to the common origin based on the additional image data, and to instruct the projector to project the visible light onto the prop based on the position of the prop relative to the common origin. The dynamic projection mapping system according to claim 7.

15. A method for operating a projection system and an optical tracking system for dynamic projection mapping, The processing circuit instructs the set of emitters of the calibration tool to emit light into the environment, The processing circuit receives image data from multiple tracking cameras indicating the position of each emitter in the set of emitters within the environment, The processing circuit instructs the projector to project visible light into the environment, The processing circuit receives sensor data indicating the visible light detected by the sensor of the calibration tool, Through the processing circuit, a common origin for the multiple tracking cameras and the projector is established within the environment based on the image data and the sensor data. Methods that include...

16. The processing circuit instructs at least three emitters of the calibration tool to emit light in the environment, The processing circuit receives initial image data from the plurality of tracking cameras, which indicates the position of each emitter in the emitter row within the environment. The processing circuit is used to calibrate the multiple tracking cameras against each other based on the initial image data, The method according to claim 15, including the method described in claim 15.

17. The method according to claim 16, comprising calibrating the plurality of tracking cameras against each other based on the initial image data before instructing the set of emitters of the calibration tool to emit light in the environment.

18. The plurality of tracking cameras and the processing circuit track the props relative to the common origin based on the detection of one or more trackers coupled to the props in the environment, The processing circuit instructs the projector to project an image onto the prop as the prop moves within the environment, based on the prop's position relative to the common origin. The method according to claim 15, including the method described in claim 15.

19. The method according to claim 15, wherein the set of emitters includes three emitters arranged to form a triangular point.

20. The method according to claim 15, wherein the calibration tool includes a rigid body having a first side and a second side opposite to the first side, the set of emitters is located on the first side and the sensors are located on the second side.

Images