Systems and methods for optical performance captured animated figure with real-time reactive projected media

Abstract

A reactive media system (8) includes a motion control system (50) having an animated figure (12) having a body and actuators configured to adjust an image portion of the body in response to interaction data received from one or more interaction data sources. The reactive media system includes a media control system (20) having a tracking camera (64) configured to generate a signal indicative of a current position and orientation of the image portion based on a set of trackers (60) coupled to the image portion. The media control system includes a media controller configured to receive the signal, determine the current position and orientation based on the signal, and generate data indicative of an image to be projected onto an outer surface of the image portion having the current position and orientation. The media control system further includes a projector (16) configured to receive the data from the media controller and project the image onto the outer surface.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and interest in U.S. Provisional Application No. 62 / 956468, filed January 2, 2020, entitled “SYSTEMS AND METHODS FOR OPTICAL PERFORMANCE CAPTURED ANIMATED FIGURE WITH REAL-TIME REACTIVE PROJECTED MEDIA,” which is hereby incorporated in its entirety by reference for all purposes. Background Technology

[0003] This section aims to introduce the reader to various technical aspects that may relate to the present technology, which are described and / or claimed below. This discussion is intended to provide the reader with background information to facilitate a better understanding of the various aspects of this disclosure. Therefore, it should be understood that these statements are to be read in this context, rather than as an admission of prior art.

[0004] Amusement parks and other entertainment venues, in addition to many other attractions, include animated characters used to entertain park patrons queuing for or already inside rides. Some animated characters can be brought to life through projection mapping, which traditionally involves projecting a predetermined appearance onto them. For example, a particular animated character can be visually complemented using a pre-recorded or fixed set of images aligned with its pre-programmed movements. While such techniques can provide more entertainment than a flat display surface, it is now recognized that enhancements can be made to further immerse patrons within specific attractions, rides, or interactive experiences. For instance, some animated characters have an internally positioned projector that generates unrealistic backlighting or glow through a translucent projection surface of the character via internal or rear projection. Accordingly, it is now recognized that the goal is to make animated characters appear more lifelike and to provide them with the ability to blend into their environment in a realistic and convincing manner according to context. Summary of the Invention

[0005] Certain embodiments corresponding in scope to the original claimed subject matter are outlined below. These embodiments are not intended to limit the scope of this disclosure, but rather are intended only to provide a brief summary of some of the disclosed embodiments. In fact, this disclosure may include a wide variety of forms that may be similar to or different from the embodiments set forth below.

[0006] In one embodiment, a reactive media system for an amusement park attraction includes a motion control system comprising an animated character having a body and a set of actuators configured to adjust an image portion of the body in response to interactive data received from one or more interactive data sources. The motion control system also includes a set of trackers coupled to the image portion of the body. The reactive media system includes a media control system comprising a tracking camera configured to generate signals indicating the current position and orientation of the image portion based on the set of trackers. The media control system includes a media controller coupled to the tracking camera and configured to receive the signals indicating the current position and orientation. The media control system is further configured to determine the current position and orientation based on the signals and generate data indicating an image to be projected onto an outer surface of the image portion having the current position and orientation. The media control system also includes a projector communicatively coupled to the media controller. The projector is configured to receive data indicating the image from the media controller and project the image onto the outer surface of the image portion having the current position and orientation.

[0007] In one embodiment, a method for operating a reactive media system at an amusement park attraction includes receiving interactive data from one or more interactive data sources at the attraction via an animated character's character controller, and generating an interactive response for the animated character based on the interactive data via the character controller. The method includes instructing actuators disposed within the animated character via the character controller to perform the interactive response by moving at least a portion of the animated character's body. The method also includes receiving sensor feedback indicating the current position and orientation of the animated character's body from a tracking camera coupled to the media controller via the media controller. The tracking camera is configured to sense the current position and orientation based on at least one tracker hidden on the animated character's body. Additionally, the method includes generating an image set via the media controller to be projected onto the outer surface of the animated character's body having the current position and orientation, and instructing a projector via the media controller to contour map the image set onto the outer surface of the body having the current position and orientation.

[0008] In one embodiment, a reactive media system for an amusement park includes one or more interactive data sources configured to generate time-varying interactive data and a motion control system including an animated character disposed within the amusement park. The animated character includes a character portion comprising movable joints. The motion control system further includes an assembly of actuators disposed within the character portion and configured to actuate the movable joints. Additionally, the motion control system includes an character controller communicatively coupled to the actuator assembly and the one or more interactive data sources. The character controller is configured to determine an interactive response of the animated character corresponding to its current state based on interactive data from the one or more interactive data sources. The interactive response includes actuation of the movable joints. The character controller is also configured to instruct the actuator assembly to perform the interactive response. Furthermore, the reactive media system includes an assembly of trackers coupled to the character portion of the animated character. The tracker assembly enables the media control system of the reactive media system to project images onto the outer surface of the character portion of the animated character in synchronization with the interactive response of the motion control system. Attached Figure Description

[0009] These and other features, aspects, and advantages of this disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings, in which the same characters denote the same parts throughout the drawings, wherein:

[0010] Figure 1 This is a schematic diagram illustrating an embodiment of an amusement park or experience with a reactive media system according to an embodiment of the present disclosure, the reactive media system being used to control images projected onto animated characters;

[0011] Figure 2 This is a schematic diagram illustrating an embodiment of a reactive media system according to an embodiment of the present disclosure, the reactive media system including a customer sensor that enables animated characters to respond to customer input;

[0012] Figure 3 According to embodiments of this disclosure Figure 1 or Figure 2 A block diagram of an embodiment of a reactive media system, such as a media control system based on detecting the movement and position of an animated character, enabling the animated character to respond to an interactive data source;

[0013] Figure 4 According to embodiments of this disclosure Figure 1 A cross-sectional schematic diagram of an embodiment of the head portion of an animated character, the head portion having a hinged jaw and a trackable tracker mounted at the rear;

[0014] Figure 5 According to embodiments of this disclosure Figure 1A cross-sectional schematic diagram of an embodiment of the head portion of an animated character, wherein the head portion has a trackable tracker mounted at the front;

[0015] Figure 6 The projection mapping according to embodiments of this disclosure is... Figure 1 A front perspective view of an embodiment of facial features on the head portion of an animated character;

[0016] Figure 7 The projection mapping according to embodiments of this disclosure is... Figure 1 A front perspective view of another embodiment of the facial features on the head portion of the animated character;

[0017] Figure 8 According to embodiments of this disclosure, a motion control system can control... Figure 1 The animated characters respond to Figure 3 A flowchart illustrating an embodiment of the process of interacting with a data source; and

[0018] Figure 9 The media control system according to embodiments of this disclosure dynamically projects images onto... Figure 1 A flowchart illustrating an embodiment of a process on an animated character that can respond to an interactive data source at any given time. Detailed Implementation

[0019] One or more specific embodiments of this disclosure will be described below. To provide a concise description of these embodiments, not all features of the actual implementation may be described in the specification. It should be appreciated that, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developer's specific goals, which may vary depending on the implementation, such as compliance with system-related constraints and business-related constraints. Furthermore, it should be appreciated that such development efforts may be complex and time-consuming, but for those skilled in the art who benefit from this disclosure, such development efforts will be nothing more than routine tasks of design, fabrication, and manufacturing.

[0020] When describing elements of various embodiments of this disclosure, the articles “a,” “an,” and “the” are intended to mean the presence of one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may be present in addition to those listed. Furthermore, it should be understood that references to “an embodiment” or “an embodiment” in this disclosure are not intended to be construed as excluding the existence of additional embodiments that also incorporate the described features.

[0021] Current embodiments relate to a reactive media system for amusement park attractions, such as those where a projector of a media control system projects images onto the outer surface of an animated character. Notably, the animated character responds to any suitable number of interactive data sources or streams, which may include the animated character's current state, control commands from an artificial intelligence (AI) engine, customer actions or passive qualities, actions of performers backstage or on stage, etc. Accordingly, the reactive media system provides customers with a dynamic and immersive experience in which the animated character is more lifelike than what was previously achievable with current technology. In particular, the animated character can appear more lifelike by projecting onto its outer surface compared to some animated character systems that project images through the translucent surface of the animated character, thereby generating an unnatural or ethereal glowing appearance. As discussed herein, the reactive media system utilizes external tracking of the animated character (e.g., via optical performance capture or optical motion capture) to dynamically generate images and provide them onto the outer surface of the animated character, thereby reducing or eliminating any unnatural backlighting. In practice, since some animated figures may include internal projectors within the figure's body, the internal projectors do not move relative to the animated figure, and therefore do not require figure or motion tracking. As an additional benefit, positioning the projector outside the animated figure allows the space within the figure to accommodate more complex or more actuators that would otherwise hinder or physically interfere with conventional internal projection mapping techniques.

[0022] More specifically, to enhance the realism of the animated character, the animated character can be equipped with a tracker that enables the tracking camera of the media control system to identify the movement, position, and orientation of the animated character in real time via optical performance capture or optical motion capture. Therefore, since the media control system can operate independently of the animated character (e.g., by not relying on position, velocity, and / or acceleration information about the actuators of the animated character), the media control system can dynamically generate and fit projected images onto the interactive animated character at a realistic frame rate (such as by presenting textures, colors, and / or movements that appear indistinguishable from the animated character) that simulates a living character. As will be understood, in some embodiments, the media control system can generate and update a skeletal model of the animated character based on feedback from the tracking camera. The skeletal model generally represents the movable parts of the animated character (e.g., its actuable joints) and is dynamically updated to represent the current three-dimensional position (e.g., including x, y, and z coordinates), orientation, and scale (e.g., the pose of the animated character) of the animated character or its parts. Therefore, the media control system utilizes a skeletal model to generate images for projection that are precisely tailored to the current position and orientation of the responsive and / or interactive animated character. Thus, the reactive media system provides a motion control system for the animated character that responds to interactive data, and a media control system that reacts to the mechanical behavior of the animated character. As discussed below, these two closed control loops, based on the optical motion capture of the animated character, provide improved system performance to deliver an engaging character presentation to the customer, regardless of the animated character's mechanical positioning.

[0023] like Figure 1 As illustrated, the reactive media system 8 of the amusement attraction 10 includes an animated character 12 that receives images 14 (e.g., projected content) from a projector 16 (e.g., an optical projector with lenses, an external projector) of the media control system 20. In this embodiment, the amusement attraction 10 is a show set having a stage ceiling 22, a stage floor 24, and scenery objects 26 disposed between the stage ceiling 22 and the stage floor 24. The show set may also include any suitable stage lighting fixture 30, such as the lighting fixtures or devices illustrated. According to current technology, multiple customers 34 can view and / or interact with the animated character 12 from the customer area 32 of the amusement attraction 10. Although the reactive media system 8 is illustrated in a stage-type environment, it should be understood that the reactive media system 8 can be used to entertain customers 34 in any suitable entertainment environment (e.g., a dark ride, an outdoor arena, an environment adjacent to the ride path of the vehicle carrying customers 34, etc.).

[0024] It is noteworthy that the projector 16 is located outside the animated character 12, thereby allowing the enclosed volume within the animated character 12 to be utilized to accommodate components other than the projector 16, such as certain actuation systems discussed below. In the illustrated embodiment, the projector 16 is positioned in front of the animated character 12 and is obscured from the view of the customer 34 by the overhang 36 of the stage ceiling 22. It should be understood that the overhang 36 could alternatively be a screen, or the attraction 10 could take any other suitable form to allow the customer 34 to view the animated character 12. In other embodiments, the projector 16 could be positioned behind the customer 34. In this embodiment, in any case, the projector 16 projects the image 14 onto the outer surface 40 of the body 42 of the animated character 12, which corresponds to the head portion 44 of the animated character 12. Thus, the media control system 20 can deliver realistic and engaging textures from a concealed position to the head portion 44, thereby providing the customer 34 with an immersive and interactive experience. In fact, as previously mentioned, compared to animated characters with internal projection, the media control system 20 of the reaction media system 8 enables the animated character 12 to appear more lifelike because the media control system 20 does not produce the unnatural dim light inside the animated character 12 that may be generated by some internally positioned projectors.

[0025] As appreciated herein, in some embodiments, the animated character 12 is part of a motion control system 50 (e.g., a character control system, a prop control system) that operates independently of the media control system 20. For example, the motion control system 50 may receive interactive data from any suitable reaction and / or interactive data source discussed in more detail below. Thus, the animated character 12 can utilize the interactive data to dynamically update or modify its interaction with the attraction 10 and / or the customers 34 within the attraction 10. It should be understood that the motion control system 50 may instruct actuators to adjust the position of any suitable component of the attraction 10 that may be viewable to the customers 34. Additionally, as referenced below... Figure 2 and Figure 3In more detail, based on suitable interactive data, the motion control system 50 can generate interactive and / or reactive responses and instruct the motion control system 50 and / or the actuators of the animated character 12 to execute interactive and / or reactive responses in real time. As described below, the reactive media system 8 can also coordinate the operation of any interactive or actuable components of the stage lighting device 30 and / or scene objects 26 to provide responses to an interactive data source coordinated with the responses of the animated character 12. Furthermore, the motion control system 50 can control actuable motion devices 66 (e.g., actuable motion bases) physically coupled to the animated character 12. The actuable motion device 66 can be any suitable motion-generating component that enables the animated character 12 to move laterally, longitudinally, and / or vertically (e.g., translate, rotate). However, it should be understood that in other embodiments, the actuable motion device 66 can be or include a suspension system and / or flight system coupled to the animated character 12 from above the stage floor 24. Therefore, the suspension and / or flight system of the actuable motion device 66 may include any suitable actuators designed to move the animated character 12 laterally, longitudinally, and / or vertically.

[0026] It is noteworthy that, in this embodiment, the motion control system 50 includes a tracker 60 (e.g., a trackable marker) positioned on the rear surface 62 of the animated character 12. As appreciated herein, the tracker 60 may be positioned on or within any suitable surface of the animated character 12 that allows the tracker 60 to be concealed from view by customers. The tracker 60 enables the tracking camera 64 of the media control system 20 to sense or resolve the position and orientation of the animated character 12 within the attraction 10, such as via optical performance capture or optical motion capture techniques. The tracking camera 64 may also be inconspicuous or concealed in any suitable manner from view by customers 34, such as by positioning it on the stage ceiling 22 or concealing it within scenery objects 26. Thus, as will be understood, without relying on position, velocity, and / or acceleration information from the actuators of the animated character 12, the projector 16 may provide images 14 onto the animated character 12 in sync with its actual current position and orientation (e.g., pose) to provide improved performance quality with reduced latency and enhanced realism. In other embodiments, the media control system 20 can verify the operation and positioning of the projector 16 based on actuator-derived information from the motion control system 50.

[0027] It should be understood that the reactive media system 8 may include any suitable number of projectors 16, trackers 60, and tracking cameras 64. For example, more than one animated character 12 may be included within a single attraction 10. Accordingly, when multiple animated characters 12 are provided for view to customers 34 from one side of the show set, the reactive media system 8 may include at least one projector 16 for each animated character 12. However, it is now recognized that, without substantial redesign or reconfiguration of the reactive media system 8, the specific infrastructure of the reactive media system 8 enables any number of animated characters 12 movable within the optical range of at least one tracking camera 64 and within the projection cone of at least one projector 16 to receive images 14. In other words, according to this disclosure, since the additional animated characters 12 are equipped with trackers 60 and placed within the attraction 10, the reactive media system 8 can easily detect and accommodate the additional animated characters 12. In other embodiments, multiple projectors 16 may be provided to deliver content to multiple sides of a single animated character 12 (e.g., an animated character 12 that can be approached by a customer 34 from multiple parts of a performance set). Additionally, some embodiments of the animated character 12 may include at least two trackers 60 to enable a tracking camera 64 to resolve the relative positions of at least two trackers 60 for efficient tracking of the animated character 12. However, it should be understood that in less complex systems, changes in the position of a single tracker 60 can also achieve the resolution of the animated character 12's position.

[0028] Figure 2The illustration shows an embodiment of a reaction media system 8 including a media control system 20 and a motion control system 50 for amusement park attraction 10, and an example of an interactive data source 70 including a customer sensor 72. In this embodiment, the customer sensor 72 collects customer input from any customer 34 within the customer area 32. As appreciated herein, customer input is a form of interactive data that can be utilized by the animated character 12 from an adaptive response to the current state of the amusement park attraction 10 or the animated character 12. In this embodiment, the customer sensor 72 can be any suitable device that collects data from or about the customer 34 as interactive data. In some embodiments, the customer input collected by the customer sensor 72 can include active and / or passive input. For example, by performing gesture recognition, the animated character 12 can respond to a visually visible gesture of the customer 34. As passive input, the animated character 12 can respond to a determined position, orientation, feature, and / or height of the customer 34. Additionally, in some embodiments, the customer sensor 72 includes physical input devices such as buttons, levers, knobs, etc. Furthermore, it should be understood that the customer sensor 72 can collect any suitable visual data, auditory data, tactile input, data from interactive customer devices (e.g., smartphones, tablets), identification information from customer armbands, input from interactive customer toys, etc.

[0029] As mentioned, based on the interaction data, the motion control system 50 can generate interactive and / or reactive responses for the animated character 12 to perform, and then instruct the actuators of the animated character 12 to perform the interactive and / or reactive responses. As a particular, non-limiting example, the motion control system 50 can determine that the interaction data includes customer input instructing a customer 34 to greet the animated character 12 physically and / or verbally from the edge of the customer area 32. The motion control system 50 can then dynamically instruct the animated character 12 to perform an interactive physical response to the greeting of the customer 34. For example, the motion control system 50 can control the animated character 12 to rotate to face the customer 34 while waving the arm of the animated character 12. As another particular example of an interactive response, the motion control system 50 can control the animated character 12 to respond to customer input by moving closer to the determined position of the customer 34 (e.g., via an actuable motion device 66 disposed under the animated character 12) and tilting the head portion 44 of the animated character 12 to provide an appearance of attention to or eye contact with the customer 34 at a specific height.

[0030] In light of the understanding of the general operation of the reactive media system 8 above, further discussion is provided herein regarding certain embodiments of specific components of the reactive media system 8. For example, Figure 3This is a block diagram of the media system 8, illustrating a media control system 20 that can operate to deliver content or images to the animated character 12 from the outside without being communicatively coupled to the motion control system 50. In other words, the media control system 20 does not directly transmit or receive communication signals from the motion control system 50. However, as discussed below, an interactive data source 70 can be communicatively coupled upstream of both the media control system 20 and the motion control system 50 to enable coordination between the media control system 20 and the motion control system 50 without communication between the control systems 20 and 50. Furthermore, in some embodiments, a network device 90 (such as a switch or hub) can be directly communicatively coupled downstream of the interactive data source 70 to facilitate efficient communication between the interactive data source 70 and the control systems 20 and 50. However, it should be understood that the network device 90 can be omitted, multiple network devices 90 can be implemented, or any other suitable data management device can be used to facilitate the delivery of data from the interactive data source 70 to the control systems 20 and 50. In the illustrated embodiment, the animated character 12 includes an image processor 100 and an image memory 104, which may collectively form all or part of the image controller 102 of the motion control system 50. The animated character 12 also includes a tracker 60 disposed on the body 42 of the animated character 12 to enable the tracking camera 64 of the media control system 20 to sense the position and orientation, or posture, of the animated character 12. The tracker 60 may be an active device capable of transmitting personalized signals to the tracking camera 64. For example, the tracker 60 may emit infrared light, electromagnetic energy, or any other suitable signal that is undetectable by the customer 34 but distinguishable by the tracking camera 64. Alternatively, the tracker 60 may be a passive device (e.g., a reflector, a colored portion) that does not emit signals and enables the tracking camera 64 to accurately distinguish passive devices from other parts of the attraction 10 and / or the animated character 12.

[0031] Furthermore, the animated character 12 is equipped with any suitable actuator 106 that enables the animated character 12 to move in a realistic and biomimetic manner (e.g., walking, translating, rotating, pivoting, lip synchonizing). The actuator 106 may include servo motors, hydraulic cylinders, linear actuators, etc., each positioned and coupled to generate relative motion between corresponding parts of the animated character 12. For example, the corresponding set of actuators 106 may be positioned to move the arms of the animated character 12, move the articulated jaw of the animated character 12, manipulate the image parts of the animated character 12 (e.g., head part 44, arm part, torso part, leg part), etc. As mentioned above, the interactive data source 70 may include any suitable data source that provides a time-varying dataset as interactive data 109. Accordingly, in some embodiments of the interactive data source 70, which includes an input sensor or customer sensor 72 (e.g., a depth camera, input device, motion sensor), the customer sensor 72 can sense the interaction between the customer and the animated character 12 and relay the interaction data indicative of the customer interaction to the character controller 102. As discussed in more detail below, in any case, based on the interaction data 109 from the interactive data source 70, the character controller 102 can instruct the actuator 106 to dynamically manipulate the animated character 12 to respond immediately and adapt to any suitable interaction data 109 that may change over time.

[0032] Interactive data source 70 (e.g., via a wireless or wired communication path) is communicatively coupled to character controller 102 and may include generating interactive data 109 and transmitting interactive data 109 to any one or more suitable components of media controller and / or character controller 102 discussed below. For example, interactive data source 70 may include a dedicated game engine or computing device that dynamically generates instructions executable by character controller 102 to cause character 12 to modify its movement within attraction 10. In some embodiments, the dedicated game engine may be embedded as a suitable module within character controller 102. In some embodiments, interactive data 109 provided by interactive data source 70 includes inputs or instructions received from: a health monitoring system of motion control system 50, a global or main control system of attraction 10, a computing device associated with technicians of attraction 10, etc. As another example, interactive data 109 may additionally or alternatively include time of day data, weather data, social media data, stock market data, vehicle location data, or any suitable data stream that may vary over time. It should be understood that the animated character 12 can therefore adaptively respond to any suitable number of interactive data sources 70. In fact, the interactive data 109 provided by the interactive data sources 70 includes any suitable flow of input information(s) that can be selectively constant and / or time-varying to enable the animated character controller 102 to dynamically control the operation of the animated character 12. Accordingly, the motion control system 50 can execute an image feedback loop that modifies or verifies the operation of the animated character 12 based on the interactive data 109.

[0033] Additionally, in some embodiments, the interaction data source 70 may include a customer sensor 72, which includes any combination of components suitable for resolving customer interactions. In practice, it is recognized that customer interactions can be sensed by any suitable optical transducer, mechanical transducer, electromechanical transducer, electromagnetic transducer, auditory transducer, pressure transducer, and / or temperature transducer, or any type of sensor. As a specific, non-limiting example, the customer sensor 72 may include any suitable number or combination of computer vision sensors (e.g., cameras), depth cameras, light detection and ranging (LIDAR) devices, motion sensors, audio recording and / or processing devices, touch capacitive sensors, light sensors, pressure or floor mat sensors, radio frequency (RF) sensors that receive uniquely identified RF signals from user wearable devices with radio frequency identification (RFID) tags, etc. That is, the customer sensor 72 may include any suitable components or transducers that generate signals based on monitoring customers 34 in customer area 32. Although illustrated as including the customer sensor 72 outside the body 42 of the animated character 12, it should be understood that other embodiments of the motion control system 50 may include all or part of the customer sensor 72 disposed within the body 42 of the animated character 12. In other embodiments, the interactive data source 70 of the motion control system 50 may exclude the customer sensor 72 and enable the animated character 12 to adaptively respond to any other suitable source(s) of interactive data 109.

[0034] As illustrated, this embodiment of the media control system 20 includes a projector 16, four tracking cameras 64, a camera network device 110, and a media controller 112. The media controller 112 is communicatively coupled to an interactive data source 70 (e.g., via a network device 90), thereby enabling the media controller 112 to dynamically respond to the physical environment of the attraction 10, such as by determining whether customer input or another source causing change in interactive data 109 has been received. Additionally, as shown in this embodiment, the media control system 20 can be communicatively isolated from the motion control system 50. That is, the motion control system 50 is independent of the media control system 20, while the media control system 20 follows or depends on the animated character 12. Thus, the media control system 20 provides the animated character 12 with operational freedom to adaptively respond to interactive data 109 substantially in real time (e.g., within microseconds or milliseconds of interaction), while the media control system 20 monitors or tracks the movement of the animated character 12 to also project immersive textures or images onto the animated character 12 substantially in real time. Accordingly, when the motion control system 50 executes an image feedback loop, the media control system 20 simultaneously executes a media feedback loop to modify or verify the image projected onto the animated character 12. To acquire information about the current position and orientation of the animated character 12, the media control system 20 uses tracking cameras 64 appropriately positioned within the attraction 10 to identify or precisely determine the trackers 60 of the animated character 12 via optical motion capture technology. The type or configuration of the tracking cameras 64 can be individually selected to correspond to the type of tracker 60. In brief, when using an active device as the tracker 60, the tracking camera 64 can be designed to receive signals from the active device to sense the position of the animated character 12. When using a passive device as the tracker 60, the tracking camera 64 can be designed to identify passive devices on the outer surface 40 of the animated character 12. The positioning of these trackers 60, together with the geometry or skeletal model of the animated character 12, facilitates coordinated projection onto the animated character 12 at different orientations.

[0035] The illustrated embodiment's tracking camera 64 is communicatively coupled to a camera network device 110, which relays signals indicative of the current three-dimensional position (e.g., including x, y, and z coordinates), orientation, and scale (e.g., the pose of the animated character 12) of the animated character 12 or portions thereof to a media controller 112. Thus, the camera network device 110 is a network switch or sensor hub that merges multiple information streams from the tracking camera 64 for efficient processing by the media controller 112. The media controller 112 includes a media processor 114 and a media memory 116, which operate together to determine, generate, and / or adjust dynamic textures or images to be overlaid onto the animated character 12 according to its current position and orientation. The media controller 112 can then instruct a projector 16 to project the dynamic image onto the animated character 12. The image can be rendered entirely on demand based on the animated character 12's current pose (e.g., position, orientation, and scale). In a less complex configuration, images can be generated by adapting a pre-recorded video stream to the current pose of the animated character 12. According to this disclosure, the media controller 112 can be any suitable media generator or game engine with significant processing power and reduced latency. It should be understood that the media controller 112 is therefore capable of generating images to be projected onto the animated character 12 substantially in real time based on sensor data received from the tracking camera 64. In practice, the media controller 112 can maintain a skeletal model or algorithm representing the animated character 12 and its actuable parts (e.g., jaw, limbs, joints). Based on the sensor data, the media controller 112 can update the skeletal model to represent the actual current position and orientation of the animated character 12, and then generate an image to be projected onto the animated character 12 having that current position and orientation.

[0036] Projector 16 may include a projector processor 120 and a projector memory 122 for facilitating the rendering of images or dynamic textures onto the animated character 12. Projector processor 120 generally receives data indicating an image from media controller 112 and then instructs a light source within projector 16 to output the image through a lens. Projector 16 may be movable or actuable to follow and align with the animated character 12, such as based on commands received from media controller 112. Alternatively, projector 16 may be stationary. In any case, media controller 112 may determine the current silhouette or shape of the target image portion of the animated character 12 to receive the projected image based on an updated skeletal model and then instruct projector 16 to provide the image onto the silhouette. It should be understood that projector 16 may therefore obscure or block portions of the animated character 12 and / or the attraction 10 that are not designed to receive the image 14, such as the animated character 12's wig, scenery objects 26 located behind the animated character 12 relative to projector 16, etc.

[0037] Processors 100, 114, and 120 are each any suitable processor capable of executing instructions for implementing the currently disclosed technology, such as a general-purpose processor, a system-on-a-chip (SoC) device, an application-specific integrated circuit (ASIC), a programmable logic controller (PLC), an industrial PC (IPC) processor, or some other similar processor configuration. These instructions are encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as memories 104, 116, 122, and / or other storage circuitry or devices. Accordingly, image processor 100 is coupled to image memory 104, media processor 114 is coupled to media memory 116, and projector processor 120 is coupled to projector memory 122.

[0038] This embodiment of the reactive media system 8 also includes a performance control system 130, which coordinates additional output devices of the attraction 10 based on the interactive data source 70. For example, the performance controller 132 of the performance control system 130 is communicatively coupled between the network device 90 and one or more lighting output devices 134, audio output devices 136, and / or location-specific special effects output devices 138 (e.g., actuable parts of the scene object 26, smoke generators, vibration generators). The lighting output device 134 may include stage lighting fixtures 30 (e.g., including lighting fixtures) and any other light-generating components of the attraction 10. Additionally, the audio output device 136 may include any suitable speaker or noise-generating device designed to output sound based on instructions from the performance controller 132. Accordingly, based on interactive data 109 received from the interactive data source 70, the performance controller 132 can coordinate the presentation of the output devices 134, 136, and 138 to correspond to the interactive and / or reactive responses of the media control system 20 and the animated character 12.

[0039] As a specific example of current technology Figure 4 This is a cross-sectional side view of the head portion 44 of the animated character 12, which has a tracker 60 on the rear surface 62 of the body 42 relative to the projector 16. In this embodiment, the tracker 60 is an active tracker (e.g., an infrared LED) and is thus connected to circuitry 150, which is coupled to or integrated with a power supply 152 of the animated character 12. The power supply 152 may be the main power supply for the animated character 12, or alternatively, a separate power supply dedicated to powering the tracker 60. Generally, the tracker 60 uses electrical energy from the power supply 152 to emit personalized signals for the tracking camera 64 to distinguish. For example, a first tracker 160 of the tracker 60 may emit a first signal having a first unique identifier (e.g., frequency, signature, signal characteristics). A second tracker 162 and a third tracker 164 of the tracker 60 may also emit signals with corresponding unique identifiers. By distributing the tracker 60 around the curved portion 166 of the head portion 44 of the body 42, the tracker 60 can enhance the trackability of the animated character 12, for example, by making the head portion 44 distinguishable from multiple sides (e.g., top side, back side) of the animated character 12 by the tracking camera 64.

[0040] Furthermore, the tracker 60 can extend from the curved portion 166 of the animated figure 12 with a protrusion distance 170. It is noteworthy that, since the signal emitted by the tracker 60 can penetrate certain materials, the tracker 60 with the protrusion distance 170 can be concealed by the costume of the animated figure 12 (e.g., wig, hat, fur) to hide the tracker 60 and further contribute to the realism of the animated figure 12. However, in the illustrated embodiment, the animated figure 12 includes a wig 171 having a gap 172 (e.g., aperture, opening, round hole) formed above the tracker 60, which further facilitates the transmission of signals from the tracker 60 to the tracking camera 64 discussed above. That is, the gap 172 can be a relatively small opening formed by the wig 171 that does not interfere with the signal, while allowing the remainder of the wig 171 to efficiently conceal the tracker 60. Additionally, although the illustration shows three trackers 60, it should be understood that any suitable number of trackers 60 can be included or hidden within the animated figure 12, such as one, two, five, ten, or more trackers 60. Alternatively, as referenced below... Figure 5 The tracker 60 discussed may be flush with or even recessed within the outer surface of the animated character 12.

[0041] In this embodiment, the animated character 12 also includes an assembly of actuators 106 for moving the head portion 44 of the animated character 12. For example, the actuators 106 may be operatively coupled to a multidirectional support 173, illustrated as a universal joint 174 (e.g., a U-joint, universal coupling device) with a pivot assembly 175, to mimic or mirror the movement of a human neck. Specifically, based on selective forces 176 provided by the actuators 106, the multidirectional support 173 can enable the head portion 44 to move along a head nodding axis 178, a head rotation axis 180, and / or a head tilt axis 182, thereby providing multiple degrees of freedom of movement to the animated character 12. It should be understood that any suitable support element or support structure with any suitable range and direction of movement may be included within the animated character 12 so as to be actuated by any suitable actuator 106, either inside or outside the animated character 12.

[0042] Furthermore, the animated character 12 of this embodiment includes a hinged jaw 184 (e.g., a movable jaw) that can move along a jaw movement axis 186, such as based on the movement of the jaw actuator 188 of the actuator 106. Accordingly, based on instructions from the character controller 102, the jaw actuator 188 can move the hinged jaw 184 along the jaw movement axis 186 (e.g., via a connected link 190 that adjusts the coupling between the pivot crank 192 on the jaw actuator 188 and the hinged jaw 184) so ​​that the animated character 12 can appear to be speaking (e.g., lip-syncing with audio output device 136), display changing facial expressions, etc. The animated character 12 may additionally or alternatively include a hinged forehead, eyebrows, ears, and / or any other movable features that contribute to the realism of a particular character or role. Furthermore, it should be understood that the articulated jaw 184 and / or any other articulated component of the animated character 12 can be manipulated based on a selective force 176 applied by any suitable single or multiple actuators 106.

[0043] Figure 5 This is a cross-sectional side view of another embodiment of the head portion 44 of the animated character 12, wherein a tracker 60 is mounted on the front surface 200 of the head portion 44. The tracker 60 may be a passive device distinguishable by the tracking camera 64. In some embodiments, the passive device is shaped as a circular cylinder or a light-emitting diode; however, it should be understood that the passive device may have any other suitable shape, including a spherical shape, a rectangular prism shape, etc. As illustrated, a first tracker 202 is positioned on the forehead portion 204 of the animated character 12, a second tracker 206 is positioned on the nose portion 208 of the animated character 12, and a third tracker 210 is positioned on the chin portion 212 of the animated character 12. By including the third tracker 210 on the chin portion 212, the animated character 12 enables the media control system 20 to efficiently analyze the movement of the articulated jaw portion 184 relative to the nose portion 208 and / or the forehead portion 204. For example, the media control system 20 can monitor the relative distance between the second tracker 206 and the third tracker 210, and thereby determine the current position of the articulated jaw 184 relative to the nose portion 208. Additionally, the animated character 12 is illustrated with a wig 171; however, it should be understood that any suitable costume element can be coupled to the animated character 12. As discussed above, when the wig 12 or other costume element is designed to cover the first tracker 202 and / or other trackers 60, the costume element can be formed with a gap 172, which enables more efficient resolution of the trackers 60 through the costume element.

[0044] The animated character 12 may also include a translucent layer 220 disposed above the body of the animated character 12, thus providing the animated character 12 with a desired appearance (e.g., skin) while allowing the tracking camera to distinguish the position of the tracker 60 through the translucent layer 220. The translucent layer 220 may be made of any suitable material (such as materials including fabric, plastic, silicone, rubber, etc.) capable of allowing the tracker 60 to be distinguished through the translucent layer 220. Accordingly, in some embodiments, the tracker 60 may be disposed under or flush with the translucent layer 220 (e.g., horizontally), thereby allowing the tracker 60 to be hidden from view by the customer. Although in Figure 4 The active device will be discussed as a tracker 60 and in Figure 5 The passive device will be discussed as tracker 60, but it should be understood that active and / or passive devices can be mounted on any suitable part of the animated character 12.

[0045] As mentioned, based on the determined actual position and orientation of the animated character 12, the media control system 20 operates the projector 16 to deliver one or more images 14 or projected content onto the outer surface 40 of the animated character 12. As a specific example, Figure 6 This is a front perspective view of image 14 provided onto the head portion 44 of the body 42 of the animated character 12. Specifically, the illustrated embodiment of image 14 includes textures similar to the face of a particular character. For example, eyebrows, eyes, nose, lips, and / or wrinkles can be projected onto the head portion 44. Furthermore, since the animated character 12 is equipped with costume elements 250 (which... Figure 6 The image is depicted as a hat, and thus the media controller 112 and / or projector 16 can identify the outline 252 of the outer surface 40 of the animated figure 12 formed by the costume element 250 (e.g., via projection masking). In other embodiments, the costume element 250 may additionally or alternatively include a wig, jewelry, scarf, etc. The projector 16 then projects the image 14 onto a target portion or image portion of the outer surface 40 of the animated figure 12. As mentioned above, and as discussed in more detail below, the media control system 20 can monitor the movement of the animated figure 12 (such as the articulated jaw 184) and project realistic images onto the head portion 44 (or its facial portion) of the animated figure 12 to correspond to a specific actual shape of the head portion 44 at any given time.

[0046] As another example, Figure 7This is a front perspective view of another embodiment of an image 14 that the projector 16 of the media control system 20 can provide onto the outer surface 40 of the animated character 12. As illustrated, the image 14 provides the animated character 12 with a character-like, non-human, or exotic appearance, such as an owl. In some embodiments, the outer surface 40 of the head portion 44 may be textured to complement the image 14. It should also be understood that in some embodiments, the image 14 may also include supernatural, exotic, or non-human images and / or effects, such as flames, smoke, transformation, color gradients, etc.

[0047] In light of the features of the reactive media system 8 described above, this document provides further discussion regarding the components of the reactive media system 8 used to entertain customers 34 via animated figures 12 having images 14 fitted thereon in real time. For example, Figure 8 This is a flowchart illustrating an embodiment of process 300, through which motion control system 50 can control animated character 12 in response to interactive data 109 from interactive data source 70. The steps illustrated in process 300 are intended to facilitate discussion and are not intended to limit the scope of this disclosure, as the illustrated steps may be performed in an alternative order or in parallel where appropriate, additional steps may be performed, and certain steps may be omitted. Process 300 may represent initiated code or instructions stored in a non-transitory computer-readable medium (e.g., character memory 104) and executed, for example, by character processor 100 of character controller 102 of motion control system 50. Character processor 100 may be communicatively coupled to other components of reactive media system 8 via a network (such as a wired or wireless network) to receive and transmit instructions and signals as described below.

[0048] In the currently illustrated embodiment, the character controller 102 of execution process 300 initiates (box 302) a performance or display by receiving (box 304) one or more suitable interactive data 109 streams from interactive data source 70, such as instructions from a health monitoring system of motion control system 50, a dedicated game engine, a global or main control system of attraction 10, a computing device associated with technicians of attraction 10, and / or sensor signals from customer sensors 72 indicating the movement or position of at least one customer 34. In practice, as mentioned above, interactive data 109 may also include time-of-day data, weather data, social media data, stock market data, transportation location data, and / or any suitable data streams that may change over time. For example, character controller 102 may receive time-of-day data from interactive data source 70 indicating that a particular character's performance should be presented. As another example, character controller 102 may receive instructions from a dedicated game engine instructing adjustments to the character's performance as indicated. As another example, the image controller 102 may receive sensor signals from the customer sensor 72, which may continuously transmit sensor signals or alternatively transmit sensor signals in response to the detection of specific movement or sound from the customer 34. In such an embodiment, the customer sensor 72 may scan the RFID tag worn by each customer 34 and transmit the identification information from the RFID tag to the image controller 102 as a sensor signal.

[0049] Furthermore, the character controller 102 can determine (box 306) one or more actions based on the interaction data 109 received from each interaction data source 70. That is, the character controller 102 can receive and respond simultaneously to multiple different interaction data sources 70, which may or may not include customer input. For example, the character controller 102 can perform appropriate animated character control actions based on interaction data 109 received from a game engine, a computing device associated with a technician at the attraction 10, a status monitoring system of the motion control system 50, etc. As a specific example, in response to receiving corresponding interaction data 109 from a computing device associated with a technician, the character controller 102 can adjust the animated character 12 from providing a detailed performance (e.g., including more than a threshold number of movements) to providing a simplified performance experience (e.g., including fewer than a threshold number of movements). As another example, in response to receiving weather data indicating an impending weather change, the character controller 102 can adjust the character or performance to interact with the changing weather (e.g., acknowledging the presence of sun, rain, and snow).

[0050] In embodiments that receive customer input, the actions determined by the image controller 102 may include determining that a customer has performed a specific action (e.g., corresponding to active input) and / or has a specific set of qualities (e.g., corresponding to passive input). As an example of a specific action detectable as customer input by the customer sensor 72, the image controller 102 may determine that a customer 34 has moved, waved, spoken, pressed a button, picked up a specific object, aimed a device (e.g., a flashlight, laser pointer, replica weapon) at a target portion of the animated image 12 within a threshold distance of the animated image 12, etc. As an example of a specific set of qualities sensed as customer input by the customer sensor 72, the image controller 102 may determine the height of each customer 34, determine the average height of a group of customers 34, retrieve profile information for each customer from a database based on RFID tag identification information, identify that a particular customer 34 is wearing clothing of a target color, etc. In practice, it should be understood that, according to current technology, the image controller 102 can determine any suitable information about the customers 34 and their specific actions in the customer area 32 to influence the interaction between the animated image 12 and the customers 34.

[0051] Furthermore, the character controller 102 can generate (box 310) an interactive response to be performed by the animated character 12 as a reaction to the actions of the attraction 10. That is, the character controller 102 can compute the interactive response to the interactive data 109 substantially in real time, thereby reacting to the current state, actions, and location of the attraction 10 and / or the customer 34 therein, to improve the customer's immersion in the attraction 10. As an example, the character controller 102 can receive input instructing a customer 34 to speak a specified name of the animated character 12 from a specific location within the customer area 32. The character controller 102 can then determine that the interactive response is for the animated character 12 to wave to the customer 34 at the specific location, move the hinged jaw 184 to correspond to the animated character saying "Hello," while the performance control system 130 causes the audio output device 136 of the attraction 10 to output the phrase "Hello."

[0052] Upon determining the interactive response, the actuator 106 of the animated character 12, following the instructions of the character controller 102 (box 312) of process 300, executes the interactive response. See below for reference. Figure 9The media control system 20, as discussed, simultaneously responds to the current pose of the animated character 12 to dynamically outline the image 14 onto the animated character 12. It should be understood that the performance controller 132 can also simultaneously control any other output devices of the attraction 10 based on the interactive data source 70 to complement the interactive response of the animated character 12. These other output devices include stage lighting fixtures 30 (e.g., lighting equipment) or other lighting output devices 134, audio output devices 136, location-specific special effects output devices 138 (e.g., smokers, air generators), etc. As an example, in response to a customer 34 pointing a flashlight at the animated character 12, the performance controller 132 can instruct the stage lighting fixtures 30 to dim, while the character controller 102 instructs the actuator 106 to cause the animated character 12 to raise its arms. As another example, in response to a customer 34 entering a specific area of ​​the customer area 32 (e.g., as sensed by a depth camera, light curtain, or pressure pad of the customer sensor 72), the image controller 102 may instruct the actuator 106 to move the animated image 12 closer to the specific area.

[0053] After executing an interactive response, the character controller 102 can return to frame 304 to continue receiving interactive data 109, allowing the motion control system 50 to continuously operate the animated character 12 during the operating hours of the attraction 10. However, it should be understood that other embodiments may implement a performance loop that adjusts or stops in response to certain predetermined conditions (such as a threshold amount of time elapsed, a threshold number of interactive responses being executed, each customer 34 leaving the customer area 32, a new or subsequent vehicle entering the customer area 32, etc.). Therefore, in such cases, the motion control system 50 maintains the wearing and operation of the animated character 12 until the next performance loop begins.

[0054] Figure 9 This is a flowchart of an embodiment of the process 350 in which the media control system 20 dynamically maps the image 14 onto the animated character 12. (See also: Regarding...) Figure 8 As mentioned in process 300, the steps of process 350 are non-limiting. Process 350 may represent startup code or instructions stored in a non-transitory computer-readable medium (e.g., media memory 116) and executed, for example, by the media processor 114 of the media controller 112 of the media control system 20. It should be understood that the media processor 114 may be communicatively coupled to other components of the reactive media system 8 via a network (such as a wired or wireless network) to receive and transmit the instructions and signals described below.

[0055] To initiate process 350 (box 352), media controller 112 may receive (box 354) signals (e.g., sensor feedback) from one or more tracking cameras 64 indicating the current pose (e.g., current position, current orientation, and / or current scale) of the animated character 12. Media controller 112 may receive signals from tracking cameras 64 via camera network device 110, facilitating efficient computation within media controller 112. As noted above, animated character 12 includes a tracker 60 hidden thereon or therein, which enables tracking cameras 64 to identify the current pose of animated character 12 without receiving or relying on position, velocity, and / or acceleration information from actuator 106 of animated character 12. Combined with the efficient scan rate of tracking cameras 64 and the frame rate of projector 16, this configuration allows animated character 12 to respond to interactive data 109 with reduced latency or lag, thereby becoming more lifelike as a living character or person.

[0056] Media controller 112 may alternatively be coupled to actuator 106 or image controller 102 and receive position, velocity, and / or acceleration information, which media controller 112 may use to verify the detected pose of animated image 12. In practice, media controller 112 may predict the future movement of animated image 12 based on the received position information, thereby providing lead time for generating projectable images. Predictively operating media controller 112 may generate multiple sets of images 14, each corresponding to a specific future interactive response of animated image 12. Accordingly, media controller 112 can thus select one of the previously generated sets of projected content, enabling media control system 20 to instantaneously provide appropriate images (e.g., textures) as a given movement sequence of animated image 12 is achieved.

[0057] In execution process 350, media controller 112 updates the skeletal model 360 of the animated character based on signals from tracking camera 64 (box 356). The skeletal model 360 may include any suitable data structures and / or statistical models maintained in media controller 112 to represent movable or actuable parts of the animated character and the current position and orientation of the actuable parts. Accordingly, media controller 112 may continuously update the skeletal model 360 to represent the actual current pose of the animated character 12 (e.g., position, orientation, and scale).

[0058] In certain situations where specific actuated joints or parts of the animated character 12 become inoperable, the media controller 112 updates the skeletal model 360 to indicate the inoperability of the specific actuated part, while reacting to the current position of the tracker 60 so that the image 14 can adaptively fit to the partially inoperable animated character 12. That is, in such cases, the media controller 112 executing process 350 can generate a suitable image corresponding to the actual pose of the animated character 12, thereby ensuring that an appropriate set of images is generated for the inoperable parts of the animated character 12. In contrast, some projection mapping systems that follow pre-programmed scenes (e.g., via pre-recorded media) can project images onto the intended positions of given movable parts of a conventional animated character, thereby degrading the customer experience when a given movable part becomes inoperable because the image does not match the conventional animated character.

[0059] Therefore, the media controller 112 generates (box 362) data based on the skeletal model 360 indicating the image 14 to be projected onto the animated character 12 with its current pose. In contrast to predetermined or pre-recorded images, the image 14 is generated in situ or in real-time to specifically correspond to the current state or pose of the animated character 12. Furthermore, the data indicating the current pose of the animated character 12 and / or the image 14 can be stored in the media memory 116 for a predetermined period of time.

[0060] Media controller 112 further instructs (box 364) projector 16 to provide (e.g., contour-map) image 14 onto the animated character 12 with its current pose. As appreciated herein, media control system 20 utilizes skeletal model 360 to perform contour mapping or contour-focus mapping to project selectively designed image 14 onto target portions of the animated character 12 in a lifelike manner. For example, media controller 112 may generate image 14 tuned to a specific position and orientation of animated character 12 and instruct projector 16 to output image 14, thereby generating a texture set of mapped contours precisely focused on the specific spatial positioning of animated character 12. Media controller 112 can inherently provide instructions for box 364 by transmitting data instructing image 14 to projector 16 without a separate instruction signal, thereby further limiting the latency of media control system 20. Accordingly, the components of the media control system 20 cooperate to adaptively analyze the animated character 12 and dynamically fit the image 14 to the instantaneous pose of the animated character 12, which can be a response to the interactive data source 70 and / or an interaction with the customer 34 at any given time. Therefore, the media controller 112 can return to box 354 to continue receiving sensor signals and continue executing process 350.

[0061] Furthermore, the media control system 20 can monitor the performance and / or health of the animated character 12. For example, it is now recognized that the media control system 20 can perform health monitoring of the animated character 12 based on its determined performance over time. That is, if a particular set of images is projected onto the animated character 12 at a specific point in time during a performance cycle (e.g., baseline performance assessment), and after a period of time, the media control system 20 adjusts (e.g., shifts) the set of images for a specific point in a later performance cycle by an adjustment exceeding a threshold (e.g., current performance assessment), the media control system 20 can generate a warning indicating the difference between iterations of the performance cycle. For example, if the head portion 44 of the animated character 12 is unintentionally modified or degraded such that the left ear of the animated character 12 is absent and not projected via the projector 16, the media control system 20 can warn the operator of the degraded condition. As another example, if a specific part of the animated character 12 moves during a first performance loop (e.g., tilting or rotating the head portion 44) but remains stationary during a second performance loop (e.g., corresponding to the modified skeletal model 360), the media control system 20 may output a warning indicating the inoperable part of the animated character 12. In these cases, the media controller 112 may output a warning indicating a deterioration state and / or a sensed non-characteristic behavioral change. In some cases, the reactive media system 8 may be designed to stop the operation of the motion control system 50 and / or the media control system 20 in response to a warning. In other cases, in response to determining that the performance of the animated character 12 is only slightly affected (e.g., having a skeletal model deviation greater than a first threshold and less than a second higher threshold), the media controller 112 may provide a warning and continue through the process 350 loop. Therefore, it should be understood that this media control system 20 can operate as a quality assessment tool for quantifying the performance of the motion control system 50 and the animated character 12 over time.

[0062] Accordingly, the technical effects of the disclosed reactive media system include a motion control system with an animated character to deliver an engaging and realistic experience to amusement park visitors, the animated character dynamically responding to interactive data, such as game engines, health monitoring systems, background performers, and input from customers and / or technicians. Specifically, the motion control system may include one or more interactive data sources providing interactive data, such as customer sensors that sense passive mass and / or active movements of customers as customer input, and an character controller that generates interactive responses based on the interactive data. Thus, the character controller instructs actuators within the animated character's body to perform interactive responses, such as by manipulating the articulated jaw or other movable parts of the body. Furthermore, the reactive media system includes a media control system with a tracking camera, a media controller, and a projector. As appreciated herein, the animated character is equipped with a trackable marker that enables the media controller to receive sensor feedback from the tracking camera indicating the current position, orientation, and scale of the movable parts of the animated character. Therefore, without relying on information derived from the actuators of the animated character, the media controller can dynamically generate or customize image sets for specific positions, orientations, and scales of the movable parts of the animated character. The media controller can then instruct the projector to project images onto the outer surface of one or more parts of the animated character in real time, thereby presenting an immersive and engagingly lifelike animated character while improving up-time and / or reliability to enhance the customer experience.

[0063] While only certain features of this disclosure have been illustrated and described herein, many modifications and alterations will occur to those skilled in the art. Therefore, it will be understood that the appended claims are intended to cover all such modifications and alterations falling within the true spirit of this disclosure. It should be appreciated that any features illustrated or described with respect to the drawings discussed above may be combined in any suitable manner.

[0064] The techniques presented and claimed herein are referenced and applied to substantial objects and specific examples of practical nature that can arguably improve the technical field and are therefore not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to this specification contains one or more elements designated as “component for [performing]...[function]” or “step for [performing]...[function]”, such elements are intended to be interpreted according to 35 USC 112(f). However, for any claim containing elements designated in any other way, such elements are not intended to be interpreted according to 35 USC 112(f).

Claims

1. A reaction media system for an amusement park, the reaction media system comprising: Motion control system, including: An animated character having a body and an actuator set configured to adjust the character portion of the body in response to interactive data received from one or more interactive data sources; and A set of trackers coupled to the image portion of the body, wherein each tracker in the set is configured to emit a personalized signal with a unique signal identified based on frequency, signature, or signal characteristics; and Media control system, including: The tracking camera is configured as follows: Based on personalized signals received from the set of trackers, the current position and current orientation of the image portion are sensed; Generate signals indicating the current position and orientation of the image portion; and A media controller, coupled to the tracking camera and configured to: Receive the signal indicating the current position and the current orientation; The current position and the current orientation are determined based on the signal; and Data is generated to indicate an image to be projected onto the outer surface of the image portion having the current position and the current orientation; and A projector communicatively coupled to the media controller, wherein the projector is configured to receive data from the media controller indicating the image, and to project the image onto the outer surface of the image portion having the current position and the current orientation.

2. The reactive media system of claim 1, comprising the one or more interactive data sources, wherein the one or more interactive data sources are configured to generate the interactive data as input data that is responsive to the current state of the animated character and is variable over time.

3. The reactive media system of claim 1, wherein the one or more interactive data sources include customer sensors configured to generate sensor signals indicative of customer input as interactive data, wherein the motion control system includes an image controller coupled to the customer sensors and the actuator assembly, wherein the image controller is configured to: The customer input is determined based on the sensor signal from the customer sensor; Generate the animated character's interactive response to the customer's input; and Instruct the actuator assembly to execute the interactive response.

4. The reactive media system of claim 3, wherein the customer sensor includes a transducer that transmits a sensor signal indicative of customer input based on monitoring the customer area of ​​the attraction.

5. The reaction media system according to claim 3, wherein the customer sensor comprises an optical transducer, a mechanical transducer, an electromechanical transducer, an electromagnetic transducer, an auditory transducer, a pressure transducer, or a temperature transducer.

6. The reactive media system of claim 3, comprising a performance controller communicatively coupled to the customer sensor and at least one output device disposed within the attraction, wherein the performance controller is configured to control the at least one output device to provide an output common to the interactive response of the animated character, and wherein the at least one output device comprises a light output device, an audio output device, or both.

7. The reactive media system of claim 1, comprising the one or more interactive data sources, the one or more interactive data sources including a customer sensor configured to detect customer input from a customer at the attraction as interactive data, and wherein the customer input includes the customer's position, movement, or action detectable by the customer sensor.

8. The reactive media system of claim 1, wherein the media control system is configured to project the image onto the outer surface of the image portion without receiving position, velocity, or acceleration information from the actuator set of the animated image.

9. The reactive media system of claim 1, wherein the image portion includes an actuated portion of a character, the actuated portion being coupled to a support structure of the character.

10. The reactive media system of claim 9, wherein the image portion includes the head portion, arm portion, torso portion, or leg portion of the body of the animated image.

11. The reactive media system of claim 1, wherein the media control system comprises a plurality of tracking cameras, the plurality of tracking cameras including the tracking camera, and wherein the plurality of tracking cameras are coupled to a camera network device, the camera network device being coupled to the media controller.

12. The reactive media system of claim 1, wherein the image portion of the animated character includes a facial portion of the body, the facial portion having a movable jaw with at least one degree of freedom, the projector is configured to project at least a portion of the image onto the movable jaw, wherein a portion of the actuator set is configured to actuate the movable jaw, and wherein the motion control system is configured to instruct the portion of the actuator set to actuate the movable jaw in response to the interactive data.

13. A method for operating a reaction media system at an amusement park attraction, the method comprising: The interactive data is received from one or more interactive data sources of the amusement park via the character controller of the animated character; The animated character's interactive response is generated based on the interactive data via the character controller. The character controller instructs actuators disposed within the animated character to perform the interactive response by moving at least a portion of the animated character's body; Via the media controller, a tracking camera coupled to the media controller receives sensor feedback indicating the current position and orientation of the body of the animated character, wherein the tracking camera is configured to sense the current position and orientation based on a personalized signal with a unique signal emitted by at least one tracker hidden on the body of the animated character, the unique signal being identified based on frequency, signature, or signal characteristics. The media controller generates a set of images to be projected onto the outer surface of the body of the animated character having the current position and orientation; as well as The media controller instructs the projector to map the outline of the image set onto the outer surface of the body having the current position and orientation.

14. The method of claim 13, wherein the one or more interactive data sources include a dedicated game engine for the animated character, a health status monitoring system for the amusement park, a main control system for the amusement park, a computing device associated with the technicians of the amusement park, or a customer sensor installed within the amusement park.

15. The method of claim 13, wherein the one or more interactive data sources include customer sensors, wherein the interactive data includes customer input indicating active movement of the customer or passive quality of the customer.

16. The method of claim 15, wherein generating the interactive response based on the interactive data comprises: Analyze the customer input to determine at least one aspect of the customer input, wherein the at least one aspect includes audible output from the customer, detectable movement of the customer, or both; and The interactive response is generated substantially in real time based on each aspect of at least one aspect of the customer input.

17. The method of claim 13, comprising: The baseline performance evaluation of the animated character is determined based on the first operation of the animated character in the first time period via the media controller. The current performance evaluation of the animated character is determined by the media controller based on the second operation of the animated character in a later second time period; The current performance assessment is compared with the baseline performance assessment via the media controller. as well as In response to determining that the current performance evaluation is outside the threshold from the baseline performance evaluation, the media controller outputs a warning indicating the deterioration state of the animated character.

18. The method of claim 13, wherein generating the image set to be projected comprises: The media controller stores a skeletal model of the animated character, including its movable parts and joints. as well as The skeletal model is updated via the media controller based on sensor feedback from the tracking camera to correspond to the current position and orientation of the animated character. as well as The set of images to be projected is generated substantially in real time based on the updated skeletal model via the media controller.

19. The method of claim 13, comprising: The media controller determines, based on sensor feedback from the tracking camera, that a specific actuator among the actuators located within the animated character is disabled. The skeletal model of the animated character stored in the media controller is updated via the media controller to indicate that the specific actuator is disabled; as well as The set of images to be projected is generated based on the updated skeletal model via the media controller.

20. A reaction media system for an amusement park, the reaction media system comprising: One or more interactive data sources configured to generate interactive data that can change over time; as well as Motion control system, comprising: An animated character, which is set within the amusement park, wherein the animated character includes a figure part, and the figure part includes movable joints; An actuator assembly disposed within the image portion and configured to actuate the movable joint; and Image controller, communicatively coupled to the actuator assembly and the one or more interactive data sources, wherein the image controller is configured to: Based on the interaction data from the one or more interaction data sources, determine an interaction response for the animated character corresponding to its current state, wherein the interaction response includes actuating the movable joints; and Instruct the actuator assembly to execute the interactive response; and A set of trackers coupled to the image portion of the animated character, wherein each tracker in the set of trackers is configured to emit a personalized signal with a unique signal identified based on frequency, signature, or signal characteristics; Media control system, including: A tracking camera, configured to sense the current position and orientation of the animated character based on personalized signals received from the set of trackers; A media controller, coupled to the tracking camera, is configured to generate content to be projected onto a portion of the animated character based on the character's current position and orientation.

21. The reactive media system of claim 20, wherein a projector is coupled to the media controller, the projector being configured to receive data indicating content from the media controller and to project the content onto the animated figure having the current position and the current orientation.

22. The reactive media system of claim 21, wherein the projector is configured to move based on instructions from the media controller to align with the current position and current orientation of the animated figure.

23. The reactive media system of claim 20, wherein the tracker set includes a first tracker disposed on the movable joint of the figure portion and a second tracker disposed on a different portion of the animated figure distinct from the movable joint, wherein the tracker set enables the media control system to determine the current distance between the different portions of the animated figure and the movable joint.

24. The reactive media system of claim 20, wherein the one or more interactive data sources include a customer sensor configured to sense input indicating interaction between a customer at the attraction and the animated character as the interactive data.

25. The reactive media system of claim 20, wherein the motion control system includes an additional animated figure with an additional figure portion, and wherein the reactive media system includes an additional set of trackers coupled to the additional figure portion to enable the media control system to project additional images onto the additional figure portion.

26. The reactive media system of claim 20, wherein the motion control system comprises an actuable motion device physically coupled to the animated character and communicatively coupled to the character controller, and wherein the interactive response comprises instructing the actuable motion device to move the animated character relative to the stage surface or ground of the attraction in at least one direction.

27. The reactive media system of claim 26, wherein... The operable motion device includes an operable motion device disposed under the animated character.

Images