System and method for optical performance capture animated figures using real-time reactive projection media

Abstract

To provide a system and a method for optical performance captured animated figure using real-time reactive projected media.SOLUTION: A reactive media system includes: a motion control system having an animated figure with a body and actuators configured to adjust a figure portion of the body in response to interactive data received from one or more interactive data sources; and a media control system having a tracking camera configured to generate signals indicative of a current position and orientation of the figure portion based on a set of trackers coupled to the figure portion. The media control system includes a media controller configured to receive the signals, determine the current position and orientation based on the signals, and generate data indicative of images to be projected onto an external surface of the figure portion having the current position and orientation. The media control system also includes media controller configured to determine the current position and orientation on the basis of the received signal and to generate data indicating an image to be projected onto an external surface of a figure part having the current position and orientation.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] 〔Cross - Reference to Related Applications〕 This application claims priority and the benefit thereof to U.S. Provisional Patent Application No. 62 / 956,468, entitled "SYSTEMS AND METHODS FOR OPTICAL PERFORMANCE CAPTURED ANIMATED FIGURE WITH REAL - TIME REACTIVE PROJECTED MEDIA", filed on January 2, 2020, the entire content of which is incorporated herein by reference for all purposes.

Background Art

[0002] This section is for introducing readers to various aspects of technologies that may be related to various aspects of the technology described and / or claimed below. This discussion is considered to be helpful in showing readers the background circumstances and promoting a better understanding of various aspects of the present disclosure. Therefore, these descriptions should not be construed as an admission of prior art, but should be understood to be read from the above perspective.

[0003] Amusement parks and other entertainment venues include, among many attractions, animated figures that entertain amusement park guests for or within ride experiences. Some animated figures can be brought to life through projection mapping, which projects a predetermined appearance onto the figure. For example, certain animated figures can be visually complemented with canned or fixed image sets that can be coordinated with the pre-programmed movements of the animated figure. While such technology offers greater entertainment value than flat display surfaces, it is now recognized that it can be further advanced to allow guests to become even more immersed in specific attractions, rides, or interactive experiences. For example, some animated figures have projectors placed inside that generate unrealistic backlighting or glow through internal or rear projection via a translucent projection surface of the animated figure. Therefore, it is now recognized that it is desirable to make animated figures look more lifelike and to provide them with the ability to harmonize with the environment contextually in a realistic and believable way. [Overview of the project] [Means for solving the problem]

[0004] The following summarizes several embodiments that fall within the same scope as the subject matter of the original claims. These embodiments are not intended to limit the scope of the disclosure, but rather to outline some of the disclosed embodiments. In practice, the disclosure may include a variety of forms that are similar to or different from the embodiments shown below.

[0005] In one embodiment, the interactive media system for an entertainment attraction includes a motion control system that includes an animated figure having a body and a set of actuators configured to adjust the figure portion of the body in response to interactive data received from one or more interactive data sources. The motion control system also includes a set of trackers coupled to the figure portion of the body. The interactive media system includes a media control system that includes a tracking camera configured to generate signals indicating the current position and orientation of the figure portion based on the set of trackers. The media control system includes a media controller coupled to the tracking camera and configured to receive signals indicating the current position and orientation. The media control system is also configured to determine the current position and orientation based on the signals and to generate data indicating an image to be projected onto the outer surface of the figure portion having the current position and orientation. The media control system also includes a projector communicatively coupled to the media controller. The projector is configured to receive image-indicating data from the media controller and project the image onto the outer surface of the figure portion having the current position and orientation.

[0006] In one embodiment, a method for operating a responsive media system for an entertainment attraction includes receiving dialogue data from one or more dialogue data sources of the entertainment attraction via a figure controller for an animated figure, and generating an interactive response for the animated figure based on the dialogue data via the figure controller. The method includes instructing actuators located within the animated figure via the figure controller to perform the interactive response by moving at least a portion of the body of the animated figure. The method also includes receiving sensor feedback from a tracking camera coupled to the media controller, via the media controller, indicating the current position and orientation of the body of the animated figure. The tracking camera is configured to sense the current position and orientation based on at least one tracker hidden on the body of the animated figure. The method also includes generating a set of images to be projected onto the outer surface of the body of the animated figure having its current position and orientation via the media controller, and instructing a projector via the media controller to contour map the set of images onto the outer surface of the body having its current position and orientation.

[0007] In one embodiment, a reactive media system for an entertainment attraction includes one or more dialogue data sources configured to generate dialogue data that changes over time, and a motion control system including an animated figure placed within the entertainment attraction. The animated figure includes figure parts including movable joints. The motion control system also includes a set of actuators placed within the figure parts and configured to actuate the movable joints. The motion control system also includes a figure controller communicatively coupled to the set of actuators and one or more dialogue data sources. The figure controller is configured to determine an interactive response for the animated figure corresponding to the current state of the animated figure, based on dialogue data from one or more dialogue data sources. The interactive response includes actinguate the movable joints. The figure controller is also configured to command the set of actuators to perform the interactive response. Furthermore, the reactive media system includes a set of trackers coupled to the figure parts of the animated figure. The set of trackers allows the media control system of the reactive media system to project images onto the outer surface of the figure parts of the animated figure in synchronization with the interactive responses of the motion control system.

[0008] A better understanding of these and other features, aspects and advantages of this disclosure will be gained by reading the following detailed description while referring to the attached drawings, which indicate the same parts throughout with the same reference numerals. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram illustrating an embodiment of an entertainment attraction or entertainment experience having a reactive media system for controlling projection mapping images displayed on an anime figure, according to an embodiment of the present disclosure. [Figure 2] This is a schematic diagram illustrating an embodiment of a reactive media system, including a guest sensor that enables an anime figure to respond to guest input, according to an embodiment of the present disclosure. [Figure 3]This is a block diagram of an embodiment of the reactive media system shown in Figure 1 or Figure 2, which enables an animated figure to respond to an interactive data source based on a media control system that detects the movement and position of the animated figure, according to an embodiment of the present disclosure. [Figure 4] This is a schematic cross-sectional view of an embodiment of the head of an animated figure shown in Figure 1, having an articulating jaw and a posteriorly attached trackable tracker, according to an embodiment of the present disclosure. [Figure 5] This is a schematic cross-sectional view of an embodiment of the head of an anime figure shown in Figure 1, having a front-mounted trackable tracker, according to an embodiment of the present disclosure. [Figure 6] This is a front perspective view of an embodiment of a facial feature projected onto the head of the anime figure in Figure 1, according to an embodiment of the present disclosure. [Figure 7] This is a front perspective view of another embodiment of a facial feature projected onto the head of the anime figure in Figure 1, according to an embodiment of the present disclosure. [Figure 8] This is a flowchart illustrating an embodiment of the process by which a motion control system, according to the embodiments of this disclosure, controls the animated figure in Figure 1 and causes it to respond to the dialogue data source in Figure 3. [Figure 9] This is a flowchart of an embodiment of the process according to the embodiments of the present disclosure in which a media control system dynamically projects an image onto an animated figure of Figure 1 that can respond to an interactive data source at any given time. [Modes for carrying out the invention]

[0010] The following describes one or more specific embodiments of this disclosure. For the sake of brevity, this specification may not describe all features of the embodiments. It should be understood that in developing any such implementation found in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developer's specific objectives, such as compliance with system-related and business-related constraints, which may vary by implementation. Furthermore, while such development efforts may be complex and time-consuming, it should be understood by those skilled in the art who benefit from this disclosure that they are routine design, fabrication, and manufacturing activities.

[0011] When describing elements of the various embodiments of this disclosure, the articles “a,” “an,” and “the” are intended to mean that there are one or more of these elements. The terms “comprising,” “including,” and “having” are intended to be comprehensive and mean that there may be further elements other than those listed. Furthermore, any reference to “one embodiment” or “a certain embodiment” in this disclosure should not be interpreted as excluding the existence of further embodiments, including the features described.

[0012] This embodiment relates to a reactive media system for entertainment attractions, such as attractions, in which a projector of a media control system directs an image onto the outer surface of an animated figure. The animated figure responds to any number of preferred interactive data sources or data streams, which may include the current state of the animated figure, control commands from an artificial intelligence (AI) engine, guest actions or passive qualities, and the actions of performers backstage or on stage. Thus, the reactive media system can provide guests with a dynamic and immersive experience in which the animated figure closely resembles a real person or character more than was possible before this technology. Specifically, by performing projection mapping onto the outer surface of the animated figure, the animated figure can appear more lifelike than some animated figure systems that produce an appearance that emits an unnatural or ethereal glow by projecting images internally through the translucent surface of the animated figure. As described herein, the reactive media system can reduce or eliminate unnatural backlighting by leveraging external tracking of the animated figure (e.g., via optical performance capture or optical motion capture) to dynamically generate and deliver images onto the outer surface of the animated figure. In practice, some anime figures can include an internal projector within their body, so the internal projector does not move relative to the anime figure and therefore does not need to track the figure or its movement. A further advantage is that by placing the projector outside the anime figure, it becomes possible to accommodate complex or numerous actuators within the space of the anime figure, which would otherwise hinder or physically interfere with conventional internal projection mapping technology.

[0013] More specifically, to enhance the naturalness of the anime figure, a tracker can be attached to the anime figure, allowing the media control system's tracking camera to identify the anime figure's movement, position, and orientation in real time via optical performance capture or optical motion capture. Since the media control system can operate independently of the anime figure (e.g., without relying on positional, velocity, and / or acceleration information related to the anime figure's actuators), it can dynamically generate and adapt projected images at a realistic frame rate to the interactive anime figure, mimicking a living character by presenting patterns, colors, and / or movements that appear indistinguishable from the anime figure. As understood, in some embodiments, the media control system can generate and update a skeletal model of the anime figure based on feedback from the tracking camera. The skeletal model generally represents the movable parts of the anime figure, such as the movable joints, and is dynamically updated to represent the current three-dimensional position, orientation, and scale (e.g., the anime figure's pose) of the anime figure or part thereof (e.g., including x, y, and z coordinates). Thus, the media control system utilizes the skeletal model to generate and project images that precisely match the current position and orientation of the reactive and / or interactive anime figure. Therefore, the responsive media system provides a motion control system having an animated figure that responds to dialogue data, and a media control system that responds to the mechanical performance of the animated figure. Thus, as will be described later, these two closed control loops provide improved system performance based on optical motion capture of the animated figure, enabling engaging character presentations to guests regardless of the mechanical positioning of the animated figure.

[0014] As shown in Figure 1, the responsive media system 8 of the entertainment attraction 10 includes an animated figure 12 that receives an image 14 (e.g., projected content) from a projector 16 (e.g., an external projector, an optical projector with a lens) of the media control system 20. In this embodiment, the entertainment attraction 10 is a show set having a stage ceiling 22, a stage floor 24, and a landscape object 26 positioned between the stage ceiling 22 and the stage floor 24. The show set may also include any preferred stage lighting device 30, such as the illustrated lighting fixtures or devices. According to this technology, multiple guests 34 can observe and / or interact with the animated figure 12 from the guest area 32 of the entertainment attraction 10. Although the responsive media system 8 is shown as existing in a stage-type environment, it should be understood that it can be used to entertain guests 34 in any preferred entertainment environment, such as a dark ride, an outdoor arena, and an environment adjacent to the ride path of a vehicle carrying guests 34.

[0015] The projector 16 is located outside the animated figure 12, and therefore, the sealed volume inside the animated figure 12 can be used to accommodate components other than the projector 16, such as several operating systems described later. In the illustrated embodiment, the projector 16 is positioned in front of the animated figure 12 and is obscured from the view of the guest 34 by an overhang 36 of the stage ceiling 22. The overhang 36 can be replaced with a curtain, or the entertainment attraction 10 can take any other preferred form that allows the guest 34 to observe the animated figure 12. In other embodiments, the projector 16 can be positioned behind the guest 34. In any case, in this embodiment, the projector 16 directs the image 14 onto the outer surface 40 of the body 42 of the animated figure 12, which corresponds to the head 44 of the animated figure 12. Therefore, the media control system 20 can supply realistic and attractive patterns to the head 44 from a hidden location, thereby providing the guest 34 with an immersive and interactive experience. In practice, as described above, the media control system 20 of the reactive media system 8 does not produce the unnatural internal illumination of the anime figure 12 that can be generated by internally placed projectors of some anime figures, and therefore can make the anime figure 12 look more lifelike than internally projected anime figures.

[0016] As recognized herein, in some embodiments, the animated figure 12 is part of a motion control system 50 (e.g., a figure control system, a prop control system) that operates independently of the media control system 20. For example, the motion control system 50 can receive dialogue data from any preferred reactive and / or interactive data source, which will be described in more detail below. Thus, the animated figure 12 can leverage the dialogue data to dynamically update or modify its interaction with the entertainment attraction 10 and / or the guest 34 inside it. It should be understood that the motion control system 50 can instruct actuators to adjust the position of any preferred component of the entertainment attraction 10 as seen by the guest 34. Furthermore, as will be described in more detail below with reference to Figures 2 and 3, the motion control system 50 can generate interactive and / or reactive responses based on preferred dialogue data and instruct the actuators of the animated figure 12 and / or the motion control system 50 to execute the interactive and / or reactive responses in real time. As will be described later, the reactive media system 8 can be coordinated to provide an interactive data source with responses that coordinate with the responses of the animated figure 12, by the operation of any interactive or operable component of the stage lighting device 30 and / or the landscape object 26. Furthermore, the motion control system 50 can control an operable motion device 66 (e.g., an operable motion base) physically coupled to the animated figure 12. The operable motion device 66 can be any preferred motion generating assembly that can move the animated figure 12 laterally, longitudinally and / or vertically (e.g., translation, rotation). However, in other embodiments, it should be understood that the operable motion device 66 may be, or include, a suspension system and / or flight system coupled to the animated figure 12 from above the stage floor 24.Accordingly, the suspension system and / or flight system of the operable motion device 66 may include any preferred actuator designed to move the animated figure 12 laterally, longitudinally, and / or vertically.

[0017] In this embodiment, the motion control system 50 includes a tracker 60 (e.g., a trackable marker) positioned on the rear surface 62 of the animated figure 12. As recognized herein, the tracker 60 can be positioned on or within any preferred surface of the animated figure 12 such that the tracker 60 is hidden from the guest's view. The tracker 60 enables the tracking camera 64 of the media control system 20 to sense or resolve the position and orientation of the animated figure 12 within the entertainment attraction 10 via optical performance capture or optical motion capture technology, etc. The tracking camera 64 can be made less visible or hidden from the guest's view in any preferred way, such as by being positioned on the stage ceiling 22 or hidden within a landscape object 26. Thus, as understood, the projector 16 can provide an image 14 to the animated figure 12 in synchronization with the actual current position and orientation (e.g., pose) of the animated figure 12, without relying on position information, velocity information and / or acceleration information from the actuators of the animated figure 12, thereby providing an improved show quality with less delay and increased realism. In another embodiment, the media control system 20 can verify the positioning and operation of the projector 16 based on actuator derivation information from the motion control system 50.

[0018] It should be understood that the responsive media system 8 may include any preferred number of projectors 16, trackers 60, and tracking cameras 64. For example, multiple animated figures 12 may be included within one entertainment attraction 10. Thus, if multiple animated figures 12 are provided that guests 34 can view from one side of the show set, the responsive media system 8 may include at least one projector 16 for each animated figure 12. However, it is now recognized that the specific infrastructure of the responsive media system 8 allows any number of animated figures 12 that are movable within the light field of at least one tracking camera 64 and movable within the projection cone of at least one projector 16 to receive an image 14, without requiring a substantial redesign or reconfiguration of the responsive media system 8. In other words, according to this disclosure, if additional animated figures 12 are placed within the entertainment attraction 10 with trackers 60, the responsive media system 8 can easily detect and accommodate these additional animated figures 12. In other embodiments, multiple projectors 16 may be provided to deliver content to multiple sides of a single animated figure 12, such as an animated figure 12 that guests 34 can approach from multiple parts of the show set. Also, while some embodiments of the animated figure 12 may include at least two trackers 60 so that a tracking camera 64 can resolve the relative positions of at least two trackers 60 for efficient tracking of the animated figure 12, it should be understood that resolving the position of the animated figure 12 can also be achieved using a less complex system by changing the position of a single tracker 60.

[0019] FIG. 2 shows an embodiment of a responsive media system 8 of an entertainment attraction 10 that includes a media control system 20 and a motion control system 50 and an example of an interaction data source 70 that includes a guest sensor 72. In this embodiment, the guest sensor 72 collects guest input from any of the guests 34 within the guest area 32. As recognized herein, guest input is one form of interaction data that the anime figure 12 can utilize to respond adaptively to the current state of the anime figure 12 or the entertainment attraction 10. In this embodiment, the guest sensor 72 can be any suitable device that collects data from or regarding the guest 34 as interaction data. In some embodiments, the guest input collected by the guest sensor 72 can include active input and / or passive input. For example, the anime figure 12 can respond to the macroscopic gestures of the guest 34 by implementing gesture recognition. The anime figure 12 can respond to the determined position, orientation, features, and / or height of the guest 34 as passive input. Also, in some embodiments, the guest sensor 72 includes physical input devices such as buttons, levers, and knobs. Further, it should be understood that the guest sensor 72 can collect any suitable visual data, auditory data, tactile input, data from interactive guest devices (e.g., smartphones, tablets), identification information from the guest's armband, and input from the guest's interactive toy, etc.

[0020] As described above, based on this dialogue data, the motion control system 50 can generate interactive and / or reactive responses that the animated figure 12 should perform, and then command the actuators of the animated figure 12 to perform the interactive and / or reactive responses. In a specific, non-limiting example, the motion control system 50 may determine that the dialogue data includes a guest input indicating that a guest 34 has physically and / or verbally greeted the animated figure 12 from the edge of the guest area 32. The motion control system 50 can then dynamically command the animated figure 12 to perform an interactive physical response to the guest 34's greeting. For example, the motion control system 50 may control the animated figure 12 to swing its arms while rotating it to face the guest 34. As another specific example of interactive response, the motion control system 50 may also control the animated figure 12 to respond to guest input by approaching the determined guest 34's position (for example, via an operable motion device 66 positioned below the animated figure 12), tilting the head 44 of the animated figure 12 to appear as if it is paying attention to or making eye contact with the guest 34 at a particular height.

[0021] Building upon an understanding of the general operation of these reactive media systems 8, this specification provides further descriptions of some embodiments of specific components of the reactive media system 8. For example, Figure 3 is a block diagram of the reactive media system 8 showing a media control system 20 that can operate to externally supply content or images to an animated figure 12 without being communicatively coupled to the motion control system 50. In other words, the media control system 20 does not directly transmit or receive communication signals to or from the motion control system 50. However, as will be discussed later, a dialogue data source 70 can be communicatively coupled upstream of both the media control system 20 and the motion control system 50 to enable coordination between the media control system 20 and the motion control system 50 without mutual communication between the control systems 20 and 50. Furthermore, in some embodiments, a network device 90 such as a switch or hub can be communicatively coupled directly downstream of the dialogue data source 70 to facilitate efficient communication between the dialogue data source 70 and the control systems 20 and 50. However, it should be understood that the network device 90 may be omitted, multiple network devices 90 may be implemented, or any other suitable data management device may be used to facilitate the supply of data from the interactive data source 70 to the control systems 20, 50. In the illustrated embodiment, the animated figure 12 includes a figure processor 100 and a figure memory 104, which together can form all or part of the figure controller 102 of the motion control system 50. The animated figure 12 also includes trackers 60 positioned on the body 42 of the animated figure 12 so that the tracking camera 64 of the media control system 20 can sense the position and orientation or pose of the animated figure 12. Each tracker 60 can be an active element capable of emitting a separate signal to the tracking camera 64. For example, the trackers 60 can emit infrared light, electromagnetic energy, or any other suitable signal that is undetectable by the guest 34 but distinguishable by the tracking camera 64.Alternatively, the tracker 60 can be a passive element (e.g., a reflector, a pigment portion) that does not emit a signal and enables the tracking camera 64 to accurately distinguish the anime figure 12 and / or other parts of the entertainment attraction 10.

[0022] Furthermore, the anime figure 12 is attached with any suitable actuator 106 that enables the anime figure 12 to move as if it were real and alive (e.g., walking, translating, rotating, pivoting, lip synchronize). The actuator 106 can include servo motors, hydraulic cylinders, linear actuators, etc. that are arranged and coupled to show the relative movement between respective parts of the anime figure 12. For example, a set of respective actuators 106 can be arranged to perform movements of the arms of the anime figure 12, movements of the articulating jaws of the anime figure 12, and operations of figure parts of the anime figure 12 (e.g., the head 44, the arm parts, the torso, the leg parts). As described above, the dialogue data source 70 can include any suitable data source that provides a data set changing over time as dialogue data 109. Thus, in some embodiments of the dialogue data source 70 including an input sensor or a guest sensor 72 (e.g., a depth camera, an input device, a motion sensor), the guest sensor 72 can sense the interaction between the guest and the anime figure 12 and relay dialogue data indicating the guest's interaction to the figure controller 102. In any case, the figure controller 102 can command the actuator 106 to dynamically operate the anime figure 12 so as to adapt immediately in response to any suitable dialogue data 109 that can change over time based on the dialogue data 109 from the dialogue data source 70, as will be described in more detail below.

[0023] A dialogue data source 70, communicably coupled to the figure controller 102 (for example, via a wireless or wired communication path), may include one or more preferred components that generate dialogue data 109 and transmit it to the figure controller 102 and / or media controllers described later. For example, the dialogue data source 70 may include a dedicated game engine or computer device that dynamically generates commands that the animated figure controller 102 can execute to correct the movements of the animated figure 12 within the amusement attraction 10. In some embodiments, the dedicated game engine can be embedded within the animated figure controller 102 as a preferred module. In some embodiments, the dialogue data 109 provided by the dialogue data source 70 may include inputs or commands received from a normal state monitoring system of the motion control system 50, a global or main control system of the amusement attraction 10, and computer devices related to the technicians of the amusement attraction 10. In addition to or instead of this, as a further example, the dialogue data 109 may also include time data, weather data, social media data, stock market data, ride location data, or any preferred data stream that can change over time. Therefore, it should be understood that the animated figure 12 can adaptively respond to any preferred number of dialogue data sources 70. In fact, the dialogue data 109 provided by the dialogue data sources 70 includes any preferred (single or multiple) streams of input information that can be selectively constant and / or change over time, so that the animated figure controller 102 can dynamically control the movement of the animated figure 12. Thus, the motion control system 50 can perform a figure feedback loop to modify or verify the movement of the animated figure 12 based on the dialogue data 109.

[0024] In some embodiments, the interaction data source 70 may include a guest sensor 72 comprising any combination of components suitable for resolving guest interactions. In fact, it is now recognized that guest interactions can be sensed by any suitable optical, mechanical, electromechanical, electromagnetic, auditory, pressure and / or temperature transducers or sensors of any kind. As a specific, non-limiting example, the guest sensor 72 may include any suitable number of computer vision sensors (e.g., cameras), depth cameras, light detection and ranging (LIDAR) devices, motion sensors, voice recording and / or processing devices, touch capacitive sensors, light sensors, pressure or floor mat sensors, radio frequency (RF) sensors that receive RF signals that uniquely identify a user wearable device having an RFID tag, or any combination thereof. That is, the guest sensor 72 may include any suitable component or transducer that generates a signal based on monitoring a guest 34 in the guest area 32. Although the motion control system 50 is shown to include a guest sensor 72 outside the body 42 of the animated figure 12, it should be understood that other embodiments of the motion control system 50 may include all or part of the guest sensor 72 located inside the body 42 of the animated figure 12. In other embodiments, the interaction data source 70 of the motion control system 50 may not include a guest sensor 72 and may allow the animated figure 12 to adaptively respond to any other preferred (one or more) source of the interaction data 109.

[0025] As shown in the figure, this embodiment of the media control system 20 includes a projector 16, four tracking cameras 64, a camera network device 110, and a media controller 112. The media controller 112 is communicatively coupled to the dialogue data source 70 (for example, via the network device 90) so that it can dynamically respond to the physical environment of the entertainment attraction 10 by determining whether a guest input or a source resulting in another change to the dialogue data 109 has been received. Also, as shown in this embodiment, the media control system 20 can be communicatively separated from the motion control system 50. That is, the motion control system 50 is independent of the media control system 20, while the media control system 20 follows or depends on the animated figure 12. Thus, the media control system 20 provides the animated figure 12 with the operational freedom to adaptively respond to the dialogue data 109 in substantially real time (for example, within microseconds or milliseconds of interaction), while monitoring or tracking the movement of the animated figure 12 and projecting immersive patterns or images onto the animated figure 12, also in substantially real time. Accordingly, the media control system 20 simultaneously runs a media feedback loop that modifies or verifies the image projected onto the animated figure 12 while the motion control system 50 is running the figure feedback loop. The media control system 20 utilizes a tracking camera 64 suitably positioned within the entertainment attraction 10 to identify or pinpoint the tracker 60 of the animated figure 12 via optical motion capture technology in order to collect information about the current position and orientation of the animated figure 12. The type or configuration of the tracking camera 64 can be individually selected to correspond to the type of tracker 60. In summary, when an active element is used as the tracker 60, the tracking camera 64 can be designed to receive a signal from the active element and sense the position of the animated figure 12. When a passive element is used as the tracker 60, the tracking camera 64 can be designed to identify the passive element on the outer surface 40 of the animated figure 12.The positioning of these trackers 60, along with the geometric or skeletal model of the anime figure 12, facilitates adjustment of projection onto the anime figure 12 in different orientations.

[0026] In the illustrated embodiment, the tracking camera 64 is communicatively coupled to a camera network device 110, which relays signals indicating the current three-dimensional position (including x, y, and z coordinates), orientation, and scale (e.g., the pose of the anime figure 12) of the anime figure 12 or a part thereof to the media controller 112. Thus, the camera network device 110 is a network switch or sensor hub that integrates multiple information streams from the tracking camera 64 for efficient processing by the media controller 112. The media controller 112 includes a media processor 114 and media memory 116 that cooperate to determine, generate, and / or adjust dynamic patterns or images to be overlaid on the anime figure 12 at its current position and orientation. As a result, the media controller 112 can instruct a projector 16 to project a dynamic image onto the anime figure 12. The image can be rendered on demand in its entirety based on the current pose (e.g., position, orientation, and scale) of the anime figure 12. In less complex configurations, the image can be generated by adapting a pre-recorded video stream to the current pose of the anime figure 12. According to this disclosure, the media controller 112 can be any preferred media generator or game engine with high processing power and low latency. Therefore, it should be understood that the media controller 112 can generate images to be projected onto the animated figure 12 in substantially real time based on sensor data received from the tracking camera 64. In fact, the media controller 112 can maintain a skeletal model or algorithm representing the animated figure 12 and its movable parts (e.g., jaw, limbs, joints). Based on the sensor data, the media controller 112 can update the skeletal model to represent the actual current position and orientation of the animated figure 12, and then generate images to be projected onto the animated figure 12 having this current position and orientation.

[0027] The projector 16 may include a projector processor 120 and a projector memory 122 to facilitate the presentation of images or dynamic patterns on the animated figure 12. The projector processor 120 generally receives image data from the media controller 112 and then commands the light source in the projector 16 to output the image through the lens. The projector 16 can be made movable or operable to follow and coordinate with the animated figure 12 based on commands received from the media controller 112, etc. Alternatively, the projector 16 can be fixed in place. In either case, the media controller 112 can determine the current silhouette or the shape of the target figure part of the animated figure 12 that is to receive the projected image, based on an updated skeletal model, and then command the projector 16 to provide the image on the silhouette. Therefore, it should be understood that the projector 16 can block or mask parts of the anime figure 12 and / or the entertainment attraction 10 that are not designed to receive the image 14 of the anime figure 12, such as the anime figure 12's wig and the landscape object 26 present behind the anime figure 12 to the projector 16.

[0028] Each of the processors 100, 114, and 120 is any preferred processor capable of executing instructions to perform the techniques of the present disclosure, such as a general-purpose processor, a system-on-a-chip (SoC) device, an application-specific integrated circuit (ASIC), a programmable logic controller (PLC) processor, an industrial PC (IPC) processor, or any other similar processor configuration. These instructions are encoded into programs or code stored in tangible, non-temporary, computer-readable media such as memories 104, 116, 122 and / or other memory circuits or storage devices. Thus, the figure processor 100 is coupled to the figure memory 104, the media processor 114 is coupled to the media memory 116, and the projector processor 120 is coupled to the projector memory 122.

[0029] This embodiment of the reactive media system 8 also includes a show control system 130 that adjusts further output devices of the entertainment attraction 10 based on an interactive data source 70. For example, the show controller 132 of the show control system 130 is communicably coupled to a network device 90 and one or more lighting output devices 134, audio output devices 136, and / or venue-specific special effects output devices 138 (e.g., a smoke machine, a vibration generator, or an operable part of a landscape object 26). The lighting output devices 134 may include stage lighting equipment 30 (e.g., including luminaires) and any other light-generating components of the entertainment attraction 10. The audio output devices 136 may also include any suitable speaker or noise generator designed to output sound based on commands from the show controller 132. Thus, the show controller 132 can adjust the presentation of the output devices 134, 136, and 138 to correspond to the interactive and / or reactive responses of the animated figures 12 and the media control system 20 based on the interactive data 109 received from the interactive data source 70.

[0030] As one specific example of this technology, Figure 4 is a side cross-sectional view of the head 44 of an animated figure 12 having a tracker 60 on the rear surface 62 of the body 42 relative to the projector 16. In this embodiment, the tracker 60 is an active tracker (e.g., an infrared light-emitting diode) and is therefore connected to a circuit 150 coupled to or integrated with the power supply 152 of the animated figure 12. The power supply 152 can be the main power supply for the animated figure 12 or a separate power supply that exclusively supplies power to the tracker 60. Generally, the tracker 60 utilizes electrical energy from the power supply 152 to emit individual signals for resolution by the tracking camera 64. For example, a first tracker 160 of the tracker 60 may emit a first signal having a first unique identification signal (e.g., frequency, signature, signal characteristics). A second tracker 162 and a third tracker 164 of the tracker 60 may also emit their respective unique identification signals. The tracker 60 can enhance the trackability of the anime figure 12 by distributing it around the curved portion 166 of the head 44 of the main body 42, allowing the tracking camera 64 to resolve the head 44 from multiple sides of the anime figure 12 (e.g., top, rear).

[0031] Furthermore, the tracker 60 can extend by a projection distance of 170 from the curved portion 166 of the anime figure 12. Since the signal emitted by the tracker 60 can pass through certain materials, a tracker 60 with a projection distance of 170 can be hidden by the costume of the anime figure 12 (e.g., wig, hat, fur) that conceals the tracker 60, further contributing to the realism of the anime figure 12. On the other hand, in the illustrated embodiment, the anime figure 12 includes a wig 171 having a gap 172 (e.g., aperture, opening, round hole) formed to cover the tracker 60, further facilitating the transmission of signals from the tracker 60 to the aforementioned tracking camera 64. That is, the gap 172 can be a relatively small opening formed through the wig 171 that does not interfere with the signal while allowing the rest of the wig 171 to efficiently conceal the tracker 60. Furthermore, although three trackers 60 are shown, it should be understood that any preferred number of trackers 60, such as one, two, five, ten, or more, can be included or hidden within the animated figure 12. Alternatively, the trackers 60 may be coplanar with the outer surface of the animated figure 12, or embedded within its outer surface, as described later with reference to Figure 5.

[0032] In this embodiment, the animated figure 12 also includes a set of actuators 106 for moving the head 44 of the animated figure 12. For example, the actuators 106 can be operably coupled to a multi-directional support 173, shown as a universal joint 174 (e.g., a U-joint, universal coupling device) having a pivot bar assembly 175, to mimic or accurately reflect the movement of a human neck. Specifically, the multi-directional support 173 can provide the animated figure 12 with multiple degrees of freedom of motion by enabling the head 44 to move along a head nod axis 178, a head swivel axis 180, and / or a head tilt axis 182 based on a selective force 176 provided by the actuators 106. It should be understood that the animated figure 12 may include any suitable support element or support structure having any suitable range of motion and direction of motion, to be actuated by any suitable actuator 106 either inside or outside the animated figure 12.

[0033] Furthermore, the animated figure 12 of this embodiment includes an articulated jaw 184 (e.g., a movable jaw) that can move along the jaw movement axis 186 based on movements such as those by a jaw actuator 188 among the actuators 106. Thus, the jaw actuator 188 can move the articulated jaw 184 along the jaw movement axis 186 based on commands from the figure controller 102 (e.g., via adjustment of a connecting link 190 coupled between the articulated jaw 184 and a pivot crank 192 on the jaw actuator 188) to make the animated figure 12 appear to be speaking (e.g., moving its mouth in sync with the sound of an audio output device 136), and to display various facial expressions, etc. In addition to or instead of this, the animated figure 12 may also include an articulated forehead, eyebrows, ears and / or any other movable features that contribute to the realism of a particular person or character. Furthermore, it should be understood that the articulated jaw 184 and / or any other articulated components of the animated figure 12 can be operated based on a selective force 176 applied by any preferred single or multiple actuators 106.

[0034] Figure 5 is a side cross-sectional view of another embodiment of the head 44 of an animated figure 12, with a tracker 60 mounted on the front surface 200 of the head 44. The tracker 60 can be a passive element distinguishable by a tracking camera 64. In some embodiments, the passive element is molded as a rounded cylinder or light-emitting diode, but it should be understood that the passive element may also have any other preferred shape, including spherical and rectangular prism shapes. As shown, a first tracker 202 is positioned on the forehead portion 204 of the animated figure 12, a second tracker 206 is positioned on the nose portion 208 of the animated figure 12, and a third tracker 210 is positioned on the jaw portion 212 of the animated figure 12. By including the third tracker 210 on the jaw portion 212, the animated figure 12 allows the media control system 20 to efficiently analyze the movement of the articulated jaw 184 relative to the nose portion 208 and / or forehead portion 204. For example, the media control system 20 can monitor the relative distance between the second tracker 206 and the third tracker 210 and determine the current position of the articular jaw 184 relative to the nose portion 208 from this distance. Also, although the anime figure 12 is shown to include a wig 171, it should be understood that any suitable costume element can be attached to the anime figure 12. As described above, if the wig 12 or other costume element is designed to cover the first tracker 202 and / or other trackers 60, these costume elements can form gaps 172 that allow for more efficient resolution of the trackers 60 through the costume element.

[0035] The anime figure 12 may also include a translucent layer 220 positioned to cover the body of the anime figure 12, thereby providing the anime figure 12 with a desired appearance (e.g., skin) while allowing a tracking camera to resolve the position of the tracker 60 through the translucent layer 220. The translucent layer 220 can be formed of any suitable material that allows for the resolution of the tracker 60 through the translucent layer 220, such as materials including cloth, plastic, silicone, and rubber. Thus, in some embodiments, the tracker 60 can be hidden from the guest's view by positioning it below the translucent layer 220 or on the same plane (e.g., at the same height) as the translucent layer 220. Although Figure 4 describes the tracker 60 as having an active element and Figure 5 describes the tracker 60 as having a passive element, it should be understood that the active element and / or passive element can be attached to any suitable part of the anime figure 12.

[0036] As described above, the media control system 20 operates the projector 16 to supply one or more images 14 or projection content onto the outer surface 40 of the anime figure 12 based on the determined actual position and orientation of the anime figure 12. As a particular example, Figure 6 is a front perspective view of an image 14 provided on the head 44 of the body 42 of the anime figure 12. Specifically, exemplary embodiments of the image 14 include patterns that resemble the face of a particular character. For example, eyebrows, eyes, nose, lips and / or wrinkles can be projected onto the head 44. Furthermore, in Figure 6, the anime figure 12 is wearing a costume element 250 shown as a hat, so the media controller 112 and / or projector 16 can identify the contour 252 of the outer surface 40 of the anime figure 12 formed by the costume element 250 (e.g., via projection masking). In addition to or instead of this, in other embodiments, the costume element 250 may include a wig, jewelry and a scarf, etc. At this time, the projector 16 directs the image 14 onto the target portion or figure portion of the outer surface 40 of the anime figure 12. As described above and as will be explained in more detail below, the media control system 20 can monitor the movement of the joint jaw 184 and other parts of the anime figure 12 and project a realistic image onto the head 44 (or face portion) of the anime figure 12 that corresponds to a specific actual shape of the head 44 at any given time.

[0037] As another example, Figure 7 is a front perspective view of another embodiment of an image 14 that a projector 16 of a media control system 20 can provide onto the outer surface 40 of an animated figure 12. As shown, the image 14 provides the animated figure 12 with a character, non-human, or fantastical appearance, such as the appearance of an owl. In some embodiments, the outer surface 40 of the head 44 can be textured to complement the image 14. It should also be understood that in some embodiments, the image 14 may include supernatural, fantastical, or non-human images and / or effects such as flames, smoke, deformation, and color morphing.

[0038] Building upon these characteristics of the reactive media system 8, this specification provides further details on the operation of the components of the reactive media system 8 that entertain guests 34 through animated figures 12 to which images 14 are adapted in real time. For example, Figure 8 is a flowchart showing an embodiment of process 300 in which a motion control system 50 controls the animated figures 12 to respond to dialogue data 109 from an dialogue data source 70. The steps shown in process 300 are for illustrative purposes only, and further steps may be performed, certain steps may be omitted, and the illustrated steps may be performed in a different order or in parallel where appropriate, so this is not intended to limit the scope of the disclosure. Process 300 may represent an initiation code or instruction stored in a non-temporary computer-readable medium (e.g., figure memory 104) and executed by a figure processor 100 of a figure controller 102 of the motion control system 50, etc. The figure processor 100 may be communicably coupled to other components of the reactive media system 8 via a network, such as a wired or wireless network, to send and receive instructions and signals as described later.

[0039] In the embodiments currently shown, the figure controller 102, which performs process 300, starts a show or display by receiving one or more preferred streams of interaction data 109 from the interaction data source 70 (block 304), such as commands from a computer device related to a dedicated game engine, a normal state monitoring system for the motion control system 50, a global or main control system for the entertainment attraction 10, and / or sensor signals from a guest sensor 72 indicating the movement or position of at least one guest 34. In practice, as described above, the interaction data 109 may also include time data, weather data, social media data, stock market data, ride location data, and / or any preferred data stream that can change over time. For example, the figure controller 102 may receive time data from the interaction data source 70 indicating that a particular character performance should be presented. As another example, the figure controller 102 may receive commands from the dedicated game engine indicating commands to adjust character performances. As yet another example, the figure controller 102 may receive sensor signals from a guest sensor 72 that can transmit sensor signals continuously or in response to detecting a particular movement or sound from a guest 34. In this embodiment, the guest sensor 72 can scan the RFID tags attached by each guest 34 and transmit the identification information from the RFID tags as a sensor signal to the figure controller 102.

[0040] Furthermore, the figure controller 102 can determine one or more actions based on the dialogue data 109 received from each dialogue data source 70 (block 306). That is, the figure controller 102 can receive and respond to multiple different dialogue data sources 70, which may or may not include guest input. For example, the figure controller 102 can perform a suitable animated figure control action based on the dialogue data 109 received from a game engine, a computer device associated with an entertainer of the amusement attraction 10, and a state monitoring system for the motion control system 50. In a specific example, the figure controller 102 can adjust the animated figure 12 from providing a detailed show performance (including, for example, movements above a threshold) to providing a simple show experience (including, for example, movements below a threshold) in response to receiving corresponding dialogue data 109 from a computer device associated with an entertainer. In another example, the figure controller 102 can adjust the character or show performance to interact with the weather change (e.g., recognize the presence of sun, rain, or snow) in response to receiving weather data indicating that a weather change is imminent.

[0041] In embodiments that receive guest input, the actions determined by the figure controller 102 may include determining that the guest has performed a specific action (e.g., corresponding to active input) and / or that the guest possesses a specific set of qualities (e.g., corresponding to passive input). Examples of specific actions that can be detected as guest input by the guest sensor 72 include the figure controller 102 determining that the guest 34 has moved within a threshold distance from the animated figure 12, waved, spoken, pressed a button, picked up a specific object, or pointed a device (e.g., a flashlight, laser pointer, or imitation weapon) at a target area of ​​the animated figure 12. Examples of specific sets of qualities that can be perceived as guest input by the guest sensor 72 include the figure controller 102 determining the height of each guest 34, determining the average height of a group of guests 34, retrieving profile information for each guest from a database based on RFID tag identification information, and identifying that a particular guest 34 is wearing clothing of a target color. In fact, according to this technology, the figure controller 102 can determine which of the following preferred pieces of information can be determined regarding the guest 34 and the guest's specific actions within the guest area 32 that affect the interaction between the guest 34 and the animated figure 12.

[0042] Furthermore, the figure controller 102 can generate interactive responses that the animated figure 12 should perform in response to the actions of the entertainment attraction 10 (block 310). That is, by calculating interactive responses to dialogue data 109 in virtually real time, the figure controller 102 can respond to the current state, actions, and positions of the entertainment attraction 10 and / or guests 34 inside it in order to enhance the immersion of guests within the entertainment attraction 10. For example, the figure controller 102 can receive input from a specific location in the guest area 32 indicating that a guest 34 has uttered the name of an assigned animated figure 12. At this time, the figure controller 102 can determine that the interactive response is for the animated figure 12 to wave its hand towards the guest 34 at the specific location and move its articulated jaw 184 in response to the animated character saying "Hi to you too," while the show control system 130 causes the audio output device 136 of the entertainment attraction 10 to output the utterance "Hi to you too."

[0043] Once an interactive response is determined, the figure controller 102, following process 300, commands the actuator 106 of the animated figure 12 to perform the interactive response (block 312). The media control system 20 simultaneously forms an image 14 on the animated figure 12 in response to the current pose of the animated figure 12, as will be described later with reference to Figure 9. It should also be understood that the show controller 132 may also complement the interactive response of the animated figure 12 by simultaneously controlling any other output devices of the entertainment attraction 10, including the stage lighting device 30 (e.g., lighting fixtures) or other lighting output devices 134, audio output devices 136, and venue-specific special effects output devices 138 (e.g., smoke machines, air generators), based on the interactive data source 70. For example, in response to a guest 34 shining a flashlight at the animated figure 12, the show controller 132 may command the stage lighting device 30 to dim the lights, while the figure controller 102 may command the actuator 106 to raise both arms of the animated figure 12. As another example, in response to a guest 34 entering a specific area of ​​the guest area 32 (for example, detected by the light curtain, pressure mat, or depth camera of the guest sensor 72), the figure controller 102 could instruct the actuator 106 to bring the animated figure 12 closer to this specific area.

[0044] After performing an interactive response, the figure controller 102 can return to block 304 and continue receiving interactive data 109 so that the motion control system 50 can continuously operate the animated figure 12 during the operating hours of the entertainment attraction 10. However, it should be understood that other embodiments may also implement show cycles that adjust or stop in response to the fulfillment of certain predetermined conditions, such as the elapsed of a threshold time, the execution of a threshold number of interactive responses, the departure of each guest 34 from the guest area 32, or the entry of a new or subsequent ride vehicle into the guest area 32. In such cases, the motion control system 50 protects the operation and wear of the animated figure 12 until the next show cycle begins.

[0045] Figure 9 is a flowchart of an embodiment of process 350 in which the media control system 20 dynamically maps an image 14 onto an animated figure 12. As described above with respect to process 300 in Figure 8, the steps of process 350 are not limited. Process 350 can represent an initiation code or instruction stored in a non-temporary computer-readable medium (e.g., media memory 116) and executed by the media processor 114 of the media controller 112 of the media control system 20. It should be understood that the media processor 114 is communicatively coupled to other components of the reactive media system 8 via a network such as a wired or wireless network, and can send and receive instructions and signals as described later.

[0046] To initiate process 350 (block 352), the media controller 112 may receive signals (e.g., sensor feedback) from one or more tracking cameras 64 indicating the current pose of the animated figure 12 (e.g., current position, current orientation, and / or current scale) (block 354). The media controller 112 may receive signals from the tracking cameras 64 through a camera network device 110 that facilitates efficient calculations within the media controller 112. As described above, the animated figure 12 includes a tracker 60 hidden on or within the animated figure 12, which enables the tracking cameras 64 to identify the current pose of the animated figure 12 without receiving or relying on positional information, velocity information, and / or acceleration information from the actuators 106 of the animated figure 12. Such a configuration, coupled with the efficient scanning speed of the tracking cameras 64 and the frame rate of the projector 16, enables the animated figure 12 to closely resemble a living character or person by responding to dialogue data 109 with short delay times or delays.

[0047] Alternatively, the media controller 112 can be coupled to the actuator 106 or figure controller 102 to receive position information, velocity information, and / or acceleration information, and utilize this information to verify the detected pose of the animated figure 12. In fact, the media controller 112 can obtain a lead time for generating projectable images by predicting the future movements of the animated figure 12 based on the received position information. The predictively operating media controller 112 can generate multiple image sets 14, each corresponding to a specific future interactive response of the animated figure 12. Thus, the media controller 112 can select one of the previously generated sets of projection content, enabling the media control system 20 to instantly provide the appropriate image (e.g., a pattern) when a given sequence of movements of the animated figure 12 is realized.

[0048] The media controller 112, which executes process 350, updates the skeletal model 360 of the anime figure based on signals from the tracking camera 64 (block 356). The skeletal model 360 may include any preferred data structure and / or statistical model maintained within the media controller 112 that represents the movable or actuarial parts of the anime figure, as well as the current position and orientation of the actuarial parts. Thus, the media controller 112 can continuously update the skeletal model 360 to represent the actual current pose (e.g., position, orientation, and scale) of the anime figure 12.

[0049] In some situations where a particular movable joint or part of the anime figure 12 is not moving, the media controller 112 updates the skeletal model 360 to indicate the immobile state of the particular movable part, in response to the current position of the tracker 60, thereby enabling the image 14 to adaptively fit to the partially immobile anime figure 12. That is, in such situations, the media controller 112 performing process 350 can ensure that a set of images suitable for the immobile parts of the anime figure 12 is generated by generating suitable images corresponding to the actual pose of the anime figure 12. In contrast, some projection mapping systems that follow a pre-programmed scene (e.g., via recorded media) project images onto the expected positions of a given movable part of a conventional anime figure, and therefore the image does not match the conventional anime figure when the given movable part becomes immobile, which can detract from the guest experience.

[0050] Accordingly, the media controller 112 generates data (block 362) based on the skeletal model 360 indicating an image 14 to be projected onto the animated figure 12 having its current pose. The image 14 is generated on the fly or in real time to specifically correspond to the current state or pose of the animated figure 12, in contrast to predetermined or recorded images. The image 14 and / or the data indicating the current pose of the animated figure 12 can be stored in the media memory 116 for a predetermined period of time.

[0051] Furthermore, the media controller 112 instructs the projector 16 to provide an image 14 onto the animated figure 12 having its current pose (e.g., contour mapping) (block 364). As recognized herein, the media control system 20 performs contour mapping or contour-focus mapping with the skeletal model 360 to orient the selectively designed image 14 realistically onto the target figure portion of the animated figure 12. For example, the media controller 112 can generate a set of contour-mapped patterns precisely focused on a specific spatial relationship of the animated figure 12 by generating an image 14 adjusted to a specific position and orientation of the animated figure 12 and instructing the projector 16 to output the image 14. Essentially, the media controller 112 provides the instruction in block 364 by sending data indicating the image 14 to the projector 16 without any other instruction signals, thereby further limiting the delay time of the media control system 20. Accordingly, the components of the media control system 20 work together to adaptively analyze the animated figure 12 and dynamically adapt the image 14 to the instantaneous pose of the animated figure 12 so that it can respond to the dialogue data source 70 and / or interact with the guest 34 at any given time. Thus, the media controller 112 can continue to receive sensor signals back to block 354 and continue executing process 350.

[0052] Furthermore, the media control system 20 can monitor the performance and / or health of the animated figure 12. For example, it is now understood that the media control system 20 can perform health monitoring of the animated figure 12 based on the determined performance of the animated figure 12 over a certain period of time. That is, if a specific set of images is projected onto the animated figure 12 at a specific point in the show cycle (e.g., baseline performance evaluation), and after a certain period of time the media control system 20 adjusts (e.g., displaces) this set of images more than the threshold adjustment (e.g., current performance evaluation) for a specific point in the subsequent show cycle, the media control system 20 can generate an alert indicating the difference between the iterations of the show cycle. For example, if the head 44 of the animated figure 12 is unintentionally modified or degraded, resulting in the left ear of the animated figure 12 being missing and no longer being projected via the projector 16, the media control system 20 can alert the operator to the degradation. As another example, if a specific part of the animated figure 12, such as the tilt or rotation of the head 44, was moving during the first show cycle but not during the second show cycle (e.g., corresponding to the modified skeleton model 360), the media control system 20 can output an alert indicating the inactive part of the animated figure 12. In these situations, the media controller 112 can output an alert indicating a degraded state and / or a perceived change in non-characteristic behavior. In some cases, the reactive media system 8 can be designed to stop the operation of the media control system 20 and / or the motion control system 50 in response to the alert. In other cases, the media controller 112 can provide an alert in response to determining that the performance of the animated figure 12 is only slightly affected (e.g., having a skeleton model deviation higher than the first threshold but lower than the second higher threshold), and the process 350 can continue to cycle. Thus, it should be understood that the media control system 20 can function as a quality evaluation tool to quantify the performance of the motion control system 50 and the animated figure 12 over a period of time.

[0053] Accordingly, the technical effects of the responsive media system disclosed include a motion control system having an animated figure that dynamically responds to interaction data such as a game engine, a health monitoring system, background performers, and input from guests and / or technicians to provide guests of an entertainment attraction with an engaging and realistic experience. Specifically, the motion control system may include one or more interaction data sources that provide interaction data, such as guest sensors that sense the active actions and / or passive qualities of guests as guest input, and a figure controller that generates interactive responses based on the interaction data. Accordingly, the figure controller commands actuators within the body of the animated figure to perform interactive responses, such as by manipulating an articulated jaw or other movable parts of the body. Furthermore, the responsive media system includes a media control system having a tracking camera, a media controller, and a projector. As recognized herein, the animated figure is fitted with trackable markers that enable the media controller to receive sensor feedback from the tracking camera indicating the current position, orientation, and scale of the movable parts of the animated figure. Therefore, the media controller can dynamically generate image sets without relying on information derived from the actuators of the anime figure, or align these images to specific positions, orientations, and scales of the movable parts of the anime figure. The media controller can then instruct the projector to present an immersive, engaging, and lifelike anime figure for improved guest enjoyment with increased uptime and / or reliability by projecting and mapping the images onto the outer surface of one or more parts of the anime figure in real time.

[0054] While this specification illustrates and describes only a few features of the present disclosure, many modifications and changes will come to mind for those skilled in the art. Therefore, it should be understood that the appended claims are intended to include all such modifications and changes in accordance with the true spirit of this disclosure. Furthermore, it should be understood that any features illustrated or described in the above-mentioned figures can be combined in any suitable manner.

[0055] The claimed technologies described herein refer to and apply to tangible objects and specific examples of a practical nature that are not abstract, intangible, or purely theoretical, but which certainly improve the art. Furthermore, if any of the claims appended to the end of this specification contain one or more elements designated as "...means for performing [function]" or "...steps for performing [function]," such elements should be interpreted in accordance with 112(f) of the United States Patent Act. On the other hand, any claim containing elements designated in any other form should not be interpreted in accordance with 112(f) of the United States Patent Act. [Explanation of Symbols]

[0056] 8. Reactive Media Systems 10 Entertainment Attractions 12 Anime Figures 14 images 16 Projectors 20 Media Control Systems 22 Stage Ceiling 24 Stage Floor 26 Landscape Objects 30 Stage lighting equipment 32 Guest Area 34 Guests 40. The outer surface of the anime figure's body 42 Anime figure body 44 Anime figure heads 50 Motion control systems 60 Tracker 62 Rear view of anime figure 64 Tracking Cameras 66 Operable motion devices

Claims

1. It is a reactive media system for entertainment attractions, Motion control system and, Media control system and The motion control system is equipped with, An anime figure that has a figure part, A set of trackers coupled to the first surface of the figure portion, each configured to emit its own sensor signal, The media control system includes, Tracking camera and, A media controller that is communicatively coupled to the aforementioned tracking camera, A projector that is communicatively coupled to the media controller, The tracking camera includes, The respective sensor signals are received from the set of trackers. Based on the reception of the respective sensor signals, the current positions of the tracker set relative to each other, relative to the tracking camera, or both are determined. Based on the current position of the set of trackers, a state signal is generated indicating the current position and orientation of the figure portion. The media controller is configured as follows: The status signal is received from the aforementioned tracking camera. Based on the state signal, first image data is generated showing an image to be projected onto at least a second surface of the figure portion having the current position and orientation, including the outer surface, and different from the first surface. The projector is configured as follows: The media controller receives data showing the image, The image is projected onto the second surface of the figure portion having the current position and current orientation. It is configured in such a way. A reactive media system characterized by the following features.

2. The media control system is configured to project the image onto the second surface of the figure portion without receiving position information, velocity information, or acceleration information from the set of actuators of the animated figure. The reactive media system according to claim 1.

3. The aforementioned tracking camera, Based on the reception of the respective sensor signals, the subsequent positions of the tracker set relative to each other, relative to the tracking camera, or both thereof are determined. Based on the subsequent position of the set of trackers, a state signal is generated indicating the subsequent position and orientation of the figure portion. The reactive media system according to claim 1, configured as described above.

4. The aforementioned media controller Based on the current position and orientation of the figure portion, a second image data showing a plurality of candidate images is generated. Based on the subsequent position and orientation of the figure portion, a subsequent image is determined from the second image data. The reactive media system according to claim 3, configured as described above.

5. The projector is configured to project the subsequent image onto the second surface of the figure portion having the subsequent position and orientation. The reactive media system according to claim 4.

6. The set of trackers includes a set of active elements, a set of passive elements, or a combination thereof. The reactive media system according to claim 1.

7. Each tracker in the set of trackers is positioned within the surface of the figure portion of the anime figure, hidden from the guest's view, and each tracker is configured to emit its respective sensor signal through the surface so that it can be received by the tracking camera. The reactive media system according to claim 1.

8. The media control system includes a plurality of tracking cameras, including the tracking camera, and the plurality of tracking cameras are coupled to a camera network device coupled to the media controller. The reactive media system according to claim 1.

9. Each of the plurality of tracking cameras has a field of view, and the field of view of each tracking camera overlaps the set of trackers in different orientations. The reactive media system according to claim 8.

10. The plurality of tracking cameras are configured to determine the current position of the set of trackers via a registration algorithm. The reactive media system according to claim 9.

11. The motion control system includes a set of actuators configured to adjust the figure portion in response to dialogue data received from one or more dialogue data sources. The reactive media system according to claim 1.

12. The motion control system is configured to stop the movement of the figure portion in response to elapsed time, the number of interactive responses, detection of a vehicle, one or more guests leaving the location, or a combination thereof. The reactive media system according to claim 11.

13. The figure portion of the anime figure includes a face portion having a movable jaw with at least one degree of freedom, the projector is configured to project at least a portion of the image onto the movable jaw, a portion of the set of actuators is configured to operate the movable jaw, and the motion control system is configured to command the portion of the set of actuators to operate the movable jaw in response to dialogue data. The reactive media system according to claim 1.

14. It is a reactive media system for entertainment attractions, One or more dialogue data sources configured to generate dialogue data that changes over time, Motion control system and, Media control system and The motion control system is equipped with, An anime figure, including a figure part with movable joints, is placed within the aforementioned entertainment attraction. A set of actuators arranged within the figure portion and configured to operate the movable joints, A set of trackers attached to the figure portion of the aforementioned anime figure, The media control system includes, Based on the visualization of the set of trackers, the position of each tracker in the set of trackers is determined. Based on the respective positions of each tracker in the set of trackers, a signal is generated indicating the current position and orientation of the animated figure. A tracking camera configured as follows, The aforementioned tracking camera is connected in a communicative manner, The tracking camera receives the signals indicating the current position and orientation of the anime figure. Based on the current position and orientation of the anime figure, content to be projected onto a part of the anime figure is generated. A media controller configured as follows, The media controller is configured to identify non-moving parts of the animated figure based on the determined position of each tracker in the set of trackers, and to adjust the generated content based on the identified non-moving parts. A reactive media system characterized by the following features.

15. The aforementioned media controller Based on the current position and orientation of the anime figure, a skeletal model of the figure part is generated. Based on the aforementioned skeletal model, content to be projected onto a part of the anime figure is generated. The reactive media system according to claim 14, configured as described above.

16. The motion control system includes a projector configured to move in coordination with the current position and orientation of the animated figure based on commands from the media controller. The reactive media system according to claim 14.

17. The set of trackers includes a first tracker positioned on the movable joint of the figure portion and a second tracker positioned on a different part of the animated figure other than the movable joint, and the tracking camera is configured to determine a first position of the first tracker and a second position of the second tracker, and to determine the current distance between the first tracker and the second tracker based on the first position relative to the second position, and to generate the signal indicating the current position and current orientation of the animated figure based on the current distance. The reactive media system according to claim 14.

18. The motion control system comprises a figure controller that generates an interactive response based on the dialogue data, The figure controller is configured to switch from a detailed control command set to a simple control command set based on the dialogue data, wherein the detailed control command set includes a greater number of movements for the animated figure compared to the simple control command set. The reactive media system according to claim 14.

19. A method for operating a reactive media system for an entertainment attraction, The steps include receiving dialogue data from one or more dialogue data sources of the entertainment attraction via the anime figure's figure controller, The steps include generating an interactive response of the animated figure based on the dialogue data via the figure controller, The steps include: commanding one or more actuators placed within the anime figure via the figure controller to perform the interactive response by operating the movable joints of the anime figure through the operation of the one or more actuators; The steps include receiving sensor feedback indicating the current position and orientation from a tracking camera, which is coupled to a media controller and configured to sense the current position and orientation of a figure portion of the anime figure based on at least one tracker coupled to a first surface of the anime figure, via the media controller, The steps of generating data via the media controller that indicates an image to be projected onto at least a second surface, which includes the outer surface and is different from the first surface, of the figure portion having the current position and current orientation, The steps include: commanding the projector via the media controller to project the image onto the second surface of the figure portion having the current position and current orientation; A method characterized by including the following.

20. The one or more dialogue data sources include the dedicated game engine for the animated figure, the normal state monitoring system for the entertainment attraction, the main control system for the entertainment attraction, computer equipment related to the technicians of the entertainment attraction, or guest sensors placed within the entertainment attraction. The method according to claim 19.

Images