Dynamic augmented projected show elements

Abstract

SOLUTION: A method includes: determining an orientation of a mechanically animated surface relative to one or more projectors configured to project content onto the mechanically animated surface; dynamically rendering the content based on one or more parameters derived from the orientation of the mechanically animated surface relative to the one or more projectors; and presenting the content on the mechanically animated surface using the one or more projectors.EFFECT: The utilization of systems and methods described herein can improve a development process for animatronic figures, leading to a shortened deployment period of new animatronics and reduced costs associated with development of such animatronics.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This application relates to systems and methods for presenting content using projections and animatronic figures. [Background technology]

[0002] Amusement park rides can integrate sound and visual effects to enhance attractions or scenes within the ride. In some applications, animatronic figure heads, faces, and other components have been developed that combine one or more projection systems with silicone or other suitable skin materials to achieve animated, static, coordinated, and / or continuous visual effects. However, applying projections to animatronic or other robotic systems typically requires tedious manual calibration and approaches to enable alignment of the projection with the animatronic. Furthermore, many animatronics rely on mechanical actuators to generate animation or other movement, which can require long-term application of skin or other covering materials, making them costly to repair. Summary of the Invention [Means for solving the problem]

[0003] One exemplary method disclosed herein includes determining an orientation of a mechanically animated surface relative to one or more projectors configured to project content onto the mechanically animated surface; dynamically rendering the content based on one or more parameters derived from the orientation of the mechanically animated surface relative to the one or more projectors; and presenting the content on the mechanically animated surface using the one or more projectors.

[0004] In some examples, the method further includes dynamically rendering one or more movements of the mechanically animated surface, wherein the movements of the mechanically animated surface are synchronized with the content.

[0005] In some examples, the content may be dynamically rendered using vertex data and animation data.

[0006] In some examples, the orientation of the mechanically animated surface relative to the one or more projectors may be determined based on input from one or more animatronic sensors associated with the mechanically animated surface and the one or more projectors.

[0007] In some examples, the one or more parameters include one or more of an angle of projection, a ratio of projection, and a brightness of projection.

[0008] An exemplary method disclosed herein includes receiving input from at least one animatronic sensor associated with an animatronic, the animatronic including a mechanically animated surface, and deriving position information for the mechanically animated surface from the input. The method further includes dynamically rendering a next frame of content to be projected onto the mechanically animated surface based on the position information, and providing the next frame of content to at least one projector projecting content onto the mechanically animated surface.

[0009] In some examples, the method may further include dynamically rendering one or more movements of the mechanically animated surface based on the position information, the movement of the mechanically animated surface being synchronized with a next frame of the content.

[0010] In some examples, the method may further include providing one or more movements of the mechanically animated surface to a control system of the animatronic.

[0011] In some examples, the next frame of content may be dynamically rendered using the vertex data and animation data.

[0012] In some examples, the at least one animatronic sensor may be a camera.

[0013] In some examples, the method may further include receiving additional input from a sensor when the content is being projected onto the mechanically animated surface.

[0014] In some examples, the next frame of content may be dynamically rendered using a game engine.

[0015] An exemplary system disclosed herein includes one or more projectors and an animatronic including a mechanically animated surface. The system further includes a performance system in communication with the one or more projectors and the animatronic, the performance system configured to dynamically render projections of content from the one or more projectors onto the mechanically animated surface and one or more movements of the mechanically animated surface, the projections synchronized with the one or more movements of the mechanically animated surface.

[0016] In some examples, the performance system may include a real-time rendering engine in communication with a projector and with a mechanical control system of the animatronics.

[0017] In some examples, the performance system may be further configured to determine one or more movements of the mechanically animated surface.

[0018] In some examples, the performance system may be further configured to determine an orientation of the mechanically animated surface relative to the one or more projectors, and the performance system may be configured to dynamically render the projection based on the orientation of the mechanically animated surface relative to the one or more projectors.

[0019] In some examples, rendering the content may include rendering the content to be presented by two or more projectors.

[0020] In some examples, the system may include one or more animatronic sensors associated with the animatronic, and the performance system is further configured to derive positional information for the mechanically animated surface from input to the one or more animatronic sensors and dynamically render a next frame of content based on the positional information.

[0021] In some examples, the performance system may be further configured to provide the next frame of content to one or more projectors.

[0022] Additional embodiments and features are set forth in part in the description that follows, and will become apparent to those skilled in the art upon examination of the specification or may be learned by practicing the disclosed subject matter. A further understanding of the nature and advantages of the present disclosure may be obtained by reference to the remaining parts of this specification and the drawings that form a part of this disclosure. As will be appreciated by those skilled in the art, each of the various aspects and features of the present disclosure can be used to advantage in some cases individually and in some cases in combination with other aspects and features of the present disclosure. [Brief explanation of the drawings]

[0023] [Figure 1] 1 illustrates an exemplary performance system in communication with a projector and a mechanically animated surface. [Figure 2] 1 shows a schematic diagram of an exemplary performance system in communication with a projector and a mechanically animated surface. [Figure 3] 1 shows a schematic diagram of an exemplary performance system. [Figure 4] 1 is an exemplary computing system used in various examples of the present disclosure. [Figure 5] 1 is a flowchart illustrating an exemplary process for projecting content onto a mechanically animated surface. [Figure 6] 1 is a flowchart illustrating an exemplary process for adjusting the projection of content onto a mechanically animated surface based on sensor input. DETAILED DESCRIPTION OF THE INVENTION

[0024] The systems and methods described herein may be used to generate realistic animatronic characters (e.g., mobile robotic systems) and other dynamic elements. Animatronics are often used to provide dynamic and realistic visual elements in amusement park attractions and other environments. For example, animatronics are often used to create immersive environments and portray dynamic characters that can interact with guests in a realistic manner. Such animatronics can be more realistic if they can provide more detailed movements (e.g., facial movements) such as movement of eyebrows, eyes, lips, wrinkles, etc. Such detailed movements, if implemented solely by mechanical means, can use custom mechanical components that can be costly and complex to implement and maintain.

[0025] Achieving finer motion using only mechanical means (e.g., motors) can lead to higher development and maintenance costs, longer downtime, and shorter lifespans for animatronic figures. For example, motors can wear out and become unusable after extended periods of continuous use, complex mechanical systems can be expensive to design and build, and motors used in such systems can be custom-made and therefore difficult to source and replace. Furthermore, continuous mechanical motion can cause wear on other parts of the animatronic. For example, the animatronic's skin (or other elastic covering) can tear with repeated motion, leading to additional maintenance and longer downtime for the animatronic. When motors or other mechanical components are utilized for larger motions in an animatronic, lights and projections can be used in conjunction with mechanical means to achieve finer motions. Using mechanical components for larger motions can simplify the mechanical systems used in the animatronic, leading to easier implementation of the animatronic and reduced wear and tear. Such simplification of the system may therefore lead to less downtime for the animatronics.

[0026] Light can also be projected onto an animatronic figure to make the figure appear more realistic (higher fidelity). For example, light and projections can be used to provide fine details (e.g., eyebrows, shading, skin texture, etc.) that cannot be achieved by mechanical means alone. Furthermore, projections can be used to more easily provide features that would otherwise be provided by mechanical means (e.g., fine facial movements (e.g., eyebrow movement, eye movement, fine mouth movement, etc.)). Adding such features through projection is often challenging due to the complexity of aligning such projections with features on the animatronic. For example, projecting onto an animatronic is challenging because its shape is dynamic with changing topology. If the projection is not rendered for the specific movement of the animatronic, the movement of the animatronic can cause the projection to appear distorted, misaligned, or otherwise unrealistic. For example, a shadow projected onto an animatronic may appear in the correct position before the animatronic moves, but may not move with the animatronic, causing the animatronic to appear unrealistic.

[0027] Furthermore, developing an animatronic figure of a particular character is often a lengthy and expensive process. The type of data used to design and operate an animatronic is generally different from the type of data an animator uses to animate the character. Conversion of such data is often time-consuming and expensive, and can require frequent back-and-forth communication between the animatronic system developer and the animator.

[0028] The systems and methods described herein may enable the creation of high-fidelity animatronic figures using projections in conjunction with mechanical movement. For example, projections may be used to provide additional details (e.g., the appearance of blood, shadows, etc.) that enhance the fidelity of the animatronic figure that cannot be provided by mechanical means alone. Such details are properly aligned with the animatronic figure and synchronized with the animatronic figure's movements using automatic calibration between the projector and the animatronic figure and real-time dynamic rendering of the projected content. Because the content is dynamically rendered in real time or near real time, the projected content may be rendered based on the actual positioning of the projector and the animatronic figure, as well as the environmental conditions (e.g., light level, sound volume, humidity, etc.) surrounding the animatronic. Dynamic rendering further enables the creation of high-fidelity interactive animatronic figures.

[0029] The systems and methods described herein further reduce the time and costs associated with developing animatronic figures. For example, a rendering engine (e.g., a gaming engine) can utilize data types similar to those utilized by animators, thereby reducing the time and costs required for data type conversion. Similarly, the design systems described herein may use similar data types in the design of animatronics. Furthermore, the projection methods described herein allow for simpler mechanical systems in animatronics. Such systems can utilize off-the-shelf parts rather than custom parts, thereby reducing development costs as well as the cost of manufacturing such animatronics. Additionally, the design process for mechanical systems is typically shorter for simpler systems. Therefore, the systems and methods described herein can be utilized to improve the development process for animatronic figures, leading to shorter development times for new animatronics and reduced costs associated with developing such animatronics.

[0030] 1 shows a performance system 102 in communication with a projector 104 and animatronics 106, which include a mechanically animated surface 107. Broadly speaking, the performance system 102 communicates with the projector 104 and the animatronics 106 to project content onto the mechanically animated surface 107, thereby enhancing the appearance of the mechanically animated surface 107. The performance system 102 can render content in real time, for example, based on the positioning of the mechanically animated surface 107 relative to the projector 104.

[0031] The animatronics 106 may include animatronic objects, characters, etc. A mechanically animated surface 107 may be incorporated into the animatronic 106 to move in response to actuation of motors in the animatronic 106. In such examples, the mechanically animated surface 107 may form the surface of such animatronic 106. For example, the mechanically animated surface 107 may be shaped to match a particular portion of the animatronic 106. In various examples, the mechanically animated surface 107 may be a vacuum-formed material (e.g., thermoplastic, silicone, rubber, plastic, alloy, etc.) that is shaped or molded into the form of a face. However, many other shapes and forms of mechanically animated surfaces 107 may also be used. In some examples, the mechanically animated surface 107 may be further textured, colored, etc., and the projected content 105 may be adjusted to compensate for such features of the mechanically animated surface 107.

[0032] Projector 104 generally projects content 105 (e.g., images, videos, etc.) onto mechanically animated surface 107. Projector 104 may be virtually any type of imaging device or spatial light modulator configured to project light onto mechanically animated surface 107. In various examples, multiple projectors may be utilized to project content 105 onto mechanically animated surface 107 of animatronic 106.

[0033] The performance system 102 generally renders content 105 that is projected from a projector 104 onto a mechanically animated surface 107. The performance system 102 also communicates with a control system associated with the projector 104 as well as the mechanically animated surface 107. For example, the performance system 102 may communicate with the projector 104 to provide the content that is projected onto the mechanically animated surface 107. In various examples, the performance system 102 may also provide various commands to the projector 104, such as commands to adjust the positioning, field of view, focus, and / or other characteristics of the projector 104. The performance system 102 may also communicate with a control system associated with the animatronic 106 to actuate one or more motors of the animatronic 106 to move the mechanically animated surface 107.

[0034] Through communication with the projector 104 and a control system associated with the animatronics 106, the performance system 102 can coordinate the projected content 105 with the movement of the mechanically animated surface 107. For example, the movement of the animatronics 106 and / or the mechanically animated surface 107 may be determined based on the content projected onto the mechanically animated surface 107, such that the movement of the mechanically animated surface 107 is driven by the content projected onto the mechanically animated surface 107. The performance system 102 may render the projected content 105 by modifying such content based on input from various sensors associated with the animatronics 106 and / or the mechanically animated surface 107. For example, the content may be modified (e.g., bent, stretched, painted, etc.) to complement the shape and movement of the mechanically animated surface 107. Additionally, the performance system 102 can render the projected content 105 to work in coordination with a particular movement of the mechanically animated surface 107, such that the projected content 105 is synchronized with the movement of the mechanically animated surface 107. In some examples, adjustments to the projected content 105 and / or the movement of the mechanically animated surface 107 by the projector 104 may be made based on parameters such as input from various sensors, the positioning of the mechanically animated surface 107 relative to the projector 104, etc. Thus, the projected content 105 can remain in coordination with the movement of the mechanically animated surface 107, even under changing conditions.

[0035] In various examples, the performance system 102 may include or utilize one or more hosts or combinations of compute resources, which may be located, for example, on one or more servers, cloud computing platforms, computing clusters, etc. Broadly, the performance system 102 may be implemented by compute resources including hardware for memory and one or more processors. For example, the performance system 102 may utilize or include one or more processors (e.g., CPUs, GPUs, and / or programmable or configurable logic).

[0036] In some embodiments, various components of the performance system 102 may be distributed across various computing resources, and the components of the performance system 102 communicate with each other via a network and / or other communication protocols. For example, in some instances, one or more components of the performance system 102 may be implemented as a serverless service, in which case the computing resources for the various components of the performance system 102 may be located across various computing environments (e.g., cloud platforms) and may be dynamically and / or automatically reallocated (e.g., depending on resource utilization of the performance system 102). In various implementations, the performance system 102 may be implemented using an organizational processing structure (e.g., functions implemented by worker elements that are assigned compute resources, containers, virtual machines, etc.).

[0037] 1 may communicate over various networks using wired and / or wireless communication protocols. For example, the performance system 102 may be remote from the projector 104 and the animatronics 106 and may communicate with the projector 104 and the animatronics 106 using one or more wireless communication protocols. In some examples, the performance system 102 may communicate with the projector 104 and / or the animatronics 106 via a wired communication protocol. The performance system 102 may include one or more components that are integrated into or local to the projector 104 or the animatronics 106.

[0038] 2 shows a schematic diagram of a performance system 102 in communication with a projector 104 and animatronics 106. Broadly, the performance system 102 includes a rendering engine 110 in communication with various sensors 108 and a mechanical control system 112. The rendering engine 110 communicates with the projector 104 and the mechanical control system 112 to present content using the projector 104 and provide commands to move the animatronics 106. In various examples, the rendering engine 110 may utilize input from the sensors 108 in the real-time rendering of the content 105 and in moving the mechanically animated surface 107.

[0039] The rendering engine 110 may be implemented by various computing resources of the performance system 102. The rendering engine 110 is generally configured to render content in real time. For example, the rendering engine 110 may be a gaming engine (e.g., the UNREAL engine or other similar gaming engine). The real-time rendering may generally allow the content to be updated and / or changed in response to changes in the animatronic 106, the environment surrounding the animatronic 106, the position of the projector 104 relative to the animatronic 106, etc. Furthermore, the real-time rendering by the rendering engine 110 may allow the content to be projected onto the mechanically animated surface 107 by providing the content 105 aligned with the mechanically animated surface 107 based on the actual relative positioning of the projector 104 and the animatronic 106. In some examples, when multiple projectors are used to project content onto a mechanically animated surface 107, such real-time rendering may further enable blending of the content 105 between the two projectors (e.g., modifying the content based on the overlap between the two projectors to result in overlapping regions that avoid differences in color or brightness between the areas).

[0040] The animatronic sensors 108 generally provide feedback to the rendering engine 110, thereby providing input for the rendering engine 110 to render content projected by the projector 104 onto the mechanically animated surface 107 of the animatronic 106. Such feedback enables the rendering engine 110 to dynamically render content such that the projected content is properly aligned with the mechanically animated surface 107. For example, the sensors 108 enable continuous tracking of the movement of the mechanically animated surface 107 as well as the position of the mechanically animated surface 107 relative to the projector 104. The input from the sensors 108 may then be utilized by the rendering engine 110 in dynamically rendering the content projected onto the mechanically animated surface 107.

[0041] The sensors 108 may include various types of sensors, such as cameras, microphones, infrared sensors, touch interfaces, buttons, temperature sensors, light sensors, etc. The sensors 108 are generally animatronic sensors associated with the animatronic 106. That is, the sensors 108 may monitor the environment of the animatronic 106, the orientation or position of the animatronic 106, the orientation or position of the mechanically animated surface 107 of the animatronic 106, the position of the projector 104 relative to the animatronic 106, etc. For example, a light sensor may monitor ambient light in the environment of the animatronic 106. An acoustic sensor (e.g., a microphone) may monitor sounds around the animatronic 106, which in various examples includes the level of ambient noise and / or other types of acoustic input (e.g., voice) in the environment of the animatronic 106. Cameras may be used to visually locate the animatronics 106, the mechanically animated surfaces 107, and the projectors 104.

[0042] The mechanical control system 112 generally communicates with the motors 114 of the animatronics to actuate the motors 114 and move the mechanically animated surface 107 of the animatronics 106. The mechanical control system 112 may generally convert commands or data provided by the rendering engine 110 into commands configured for the motors 114 of the animatronics. For example, the rendering engine 110 may send animation data to the mechanical control system 112 using User Datagram Protocol (UDP) packets or other types of data transfer protocols. Once the mechanical control system 112 receives the animation data, it converts the animation data into motor data for the motors 114. Such motor data may include which motors to move, the direction of movement, the speed of movement, etc.

[0043] In various examples, the mechanical control system 112 may also track or monitor the actuation of the motors 114 and provide such information to the rendering engine 110 for use in rendering the content 105. For example, the relative positions of the motors 114 of the animatronic may help identify the topography of the mechanically animated surface 107 and may be used by the rendering engine 110 to render the content 105 to conform to the topography of the mechanically animated surface 107.

[0044] Although the performance system 102 is illustrated as including one rendering engine 110, in various examples, the performance system 102 includes multiple rendering engines. For example, if multiple projectors are used to project the content 105, a rendering engine for each of the multiple projectors may be used. In such examples, the rendering engines may communicate with each other to coordinate the projectors in the presentation of the content 105. For example, different projectors may be configured to project onto different regions of the mechanically animated surface 107, and the rendering engines may communicate with each other to ensure that the content projected onto the mechanically animated surface 107 is appropriately blended as a whole.

[0045] 3 shows a schematic diagram of additional components in communication with the performance system 102. This diagram illustrates various components utilized in generating content to be displayed and presenting the content using the animatronics 106 and the projectors 104. For example, content assets 116, including both vertex data 118 and animation data 120, may be provided to a design system 122, which designs the animatronics 106 and / or the content to be projected onto the animatronics 106 by the projectors 104. The design system 114 may communicate information regarding the design of the animatronics 106 (e.g., feature positions, textures, dimensions, and other characteristics of the animatronics 106) along with the vertex data 116 and animation data 120 to the performance system 102. The performance system 102 may use such data to determine the initial movement of the animatronics 106 and the mechanically animated surface 107 based on the content to be projected onto the mechanically animated surface 107. The performance system 102 may further utilize such data, along with input from the animatronic sensors 108 and (in some examples) input from the AI ​​model 128 and / or real-time puppet 130, to render content that is projected onto the mechanically animated surface 107 of the animatronic 106. In some examples, the performance system 102 may also use such input to determine further movement of the animatronic 106 and mechanically animated surface 107 that is synchronized with the projection of the content 105.

[0046] A content asset 116 may be, for example, information about an individual character or other element to be portrayed using the animatronic 106. The content asset 116 may generally be stored in one or more data stores. Such a data store may be associated, for example, with an animation studio, gaming studio, or other creator of the content asset 116. An individual content asset 116 may include vertex data 118 and animation data 120. Both types of data may be used by the design system 122 and performance system 102 to render content to be presented using the projector 104 and the animatronic 106. Broadly, the vertex data 118 is mesh data for a particular character, while the animation data 120 is data regarding how to move the mesh to animate that character. In various examples, the design system 122 uses a character's vertex data 118 and animation data 120 to design the character's animatronic 106. For example, the design system 122 may extrapolate the vertex data 118 to generate the shape of a mesh for the animatronic by performing a distortion analysis of the mesh of the vertex data 118 to determine salient points in the mesh. A shape may then be extrapolated or synthesized from those salient points. In some examples, the design system 122 may also be used to create baseline content for rendering by the performance system 102. For example, the design system 122 may be used to generate content or instructions for content, which may be rendered by the performance system 102. The design system 122 may also determine characteristics of the animatronic 106 and / or mechanically animated surface 107 based on such baseline content.For example, the animatronics 106 may be designed according to the vertex data 118, and the initial movement of the mechanically animated surfaces 107 may be determined based on the animation data 120 and / or baseline content.

[0047] The performance system 102 may receive information about the animatronic 106 and the projection from the design system 122. The performance system 102 may determine the initial movement of the animatronic 106 based on characteristics of the content to be projected onto the animatronic 106. In various examples, either before the content is rendered or when the initial content is projected onto the animatronic 106, the performance system 102 may receive additional input for rendering the content for the projector 104 and / or for generating the movement of the animatronic 106. For example, the performance system 102 may utilize an artificial intelligence (AI) model 128 to render the projected content and the movement of the animatronic 106. In some examples, a real-time puppet 130 may also provide feedback to the performance system 102 to enable user interaction with the animatronic 106.

[0048] The AI ​​model 128 may generally provide information to the performance system 102 regarding how to adjust the rendered content 105 in response to particular sensor inputs. For example, the AI ​​model 128 may be an unsupervised learning model trained using synthetic or real-world training data 124 related to sensor inputs and the rendering of projected content based on the sensor inputs. In some examples, the AI ​​model 128 may be further trained using sensor fusion data, allowing the performance system 102 to more accurately utilize inputs from multiple animatronic sensors and types of animatronic sensors at once. In some examples, the AI ​​model 128 may be further configured to identify and analyze voices, gestures, and other interactions associated with the animatronic 106. For example, microphones in the environment of the animatronic 106 may return inputs including voice, and the AI ​​model 128 can parse the voice from the input to identify questions, commands, etc. Similarly, visual sensors (e.g., cameras) in the environment of the animatronic 106 may provide visual input including user interactions with the animatronic 106, allowing the AI ​​model 128 to parse predefined actions from the user's movements.

[0049] The real-time puppet 130 may be a computing system, model, or human-operated system that enables the animatronic 106 to interact with a user in real time. For example, sensors in the performance system 102 may identify the voice of a user asking the animatronic 106 a question. The real-time puppet 130 may identify an appropriate answer to the user's question. In another example, sensors in the performance system 102 may identify gestures from a user giving specific commands to or otherwise interacting with the animatronic. The real-time puppet 130 may identify an appropriate response to the identified gesture. For example, the real-time puppet 130 may include a large-scale language model (LLM) or other model configured to generate appropriate answers to the user's questions, and the real-time puppet 130 may provide the answers to the performance system 102. Similarly, a human user may use the real-time puppet 130 to provide answers to the user's questions. Such answers may be provided to the performance system 102, which may render the content 105 and the movement of the animatronic 106 so that the animatronic 106 appears to be answering the user's question.

[0050] FIG. 4 illustrates an exemplary computing system 200 that may be used to implement various embodiments in the examples described herein. For example, in various embodiments, components of the performance system 102, the design system 122, the AI ​​model 128, and / or other components may be implemented by one or more computing systems 200. For example, the rendering engine 110 and the mechanical control system 112 may include or be implemented by a computing system 200. This disclosure contemplates any suitable number of computing systems 200. For example, the computing system 200 may be a server, a desktop computing system, a mainframe, a mesh of computing systems, a laptop or notebook computing system, a tablet computing system, an embedded computer system, a system-on-a-chip, a single-board computing system, or a combination of two or more of these. The computing system 200 may include one or more computing systems, may be single or distributed, may span multiple locations, may span multiple machines, may span multiple data centers, or may reside in a cloud, which may include one or more cloud components on one or more networks.

[0051] Computing system 200 may include a bus 210 (e.g., an address bus and a data bus) or other communication mechanism for transmitting information, which interconnects subsystems and devices, such as a processor 208, memory 202 (e.g., RAM), static storage 204 (e.g., ROM), dynamic storage 206 (e.g., magnetic or optical), a communication interface 216 (e.g., a modem, an Ethernet card, a network interface controller (NIC) or network adapter for communicating over an Ethernet or other wired network, a wireless NIC (WNIC) or wireless adapter for communicating over a wireless network such as a Wi-Fi network), and an input / output (I / O) interface 220 (e.g., a keyboard, keypad, mouse, microphone). In particular embodiments, computing system 200 may include one or more of any such components.

[0052] In particular embodiments, processor 208 includes hardware for executing instructions (e.g., instructions making up a computer program). For example, processor 208 may execute instructions for various components of a performance system. The circuitry of processor 208 includes circuitry for performing various processing functions (e.g., circuitry for executing particular software to perform particular calculations or tasks). In particular embodiments, I / O interface 220 includes hardware, software, or both that provide one or more interfaces for communication between computing system 200 and one or more I / O devices. Computing system 200 may include one or more of these I / O devices, as appropriate. One or more of these I / O devices may enable communication between a person and computing system 200.

[0053] In particular embodiments, communication interface 216 includes hardware, software, or both that provide one or more interfaces for communications (e.g., packet-based communications, etc.) between computing system 200 and one or more other computer systems or one or more networks. One or more memory buses (which may each include an address bus and a data bus) may couple processor 208 to memory 202. Bus 210, as described below, may include one or more memory buses. In particular embodiments, one or more memory management units (MMUs) reside between processor 208 and memory 202 to facilitate accesses to memory 202 requested by processor 208. In particular embodiments, bus 210 includes hardware, software, or both that couple components of computing system 200 to one another.

[0054] According to particular embodiments, computing system 200 performs certain operations by processor 208 executing one or more sequences of one or more instructions stored in memory 202. For example, instructions for various components of performance system 102 may be stored in memory 202 and executed by processor 208. Such instructions may be loaded into memory 202 from another computer-readable / usable medium (e.g., static storage 204 or dynamic storage 206). In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions. Thus, particular embodiments are not limited to any particular combination of hardware circuitry and / or software. In various embodiments, the term "logic" refers to any combination of software or hardware used to implement all or a portion of a particular embodiment disclosed herein.

[0055] The terms "computer-readable medium" or "computer-usable medium" as used herein refer to any medium that participates in providing instructions to processor 208 for execution. Such media may take many forms, including but not limited to, non-volatile media and volatile media. Non-volatile media include, for example, optical or magnetic disks, such as static storage 204 or dynamic storage 206. Volatile media include dynamic memory, such as memory 202.

[0056] The computing system 200 may send and receive messages, data, and instructions (including programs (e.g., application code)) via the communications link 218 and the communications interface 216. The received program code may be executed by the processor 208 as it is received and / or may be stored in the static storage 204, the dynamic storage 206, or other storage for later execution. A database 214 may be used to store data, which the computing system 200 can access via the data interface 212. For example, projection settings, content, vertex data, animation data, etc. may be stored in the database 214. In various examples, the communications link 218 in the performance system 102 may communicate with the projector 104, the animatronics 106, and / or other computing components within a network.

[0057] 5 illustrates a process 300 for projecting content onto the mechanically animated surface 107 of an animatronic 106. Process 300 generally involves calibrating the projector 104 and the mechanically animated surface 107 of the animatronic 106. Such calibration allows the content 105 to be properly aligned with the mechanically animated surface 107, thereby improving the appearance of the animatronic 106 and the fidelity of the combination of the content 105 and the movements of the animatronic 106 presented by the projector 104. For example, the projector 104 may be configured to project forehead wrinkles onto the mechanically animated surface 107 as the mechanically animated surface 107 moves while the character is speaking. Without calibration between the projector 104 and the mechanically animated surface 107, the wrinkles may appear in an unnatural place on the animatronic 106. Using method 300, such features are projected in the correct locations on the animatronic 106, thereby increasing the fidelity and realism of the appearance of the animatronic 106.

[0058] The calibration illustrated in process 300 may occur before content is projected onto the mechanically animated surface 107 of the animatronic 106. In some examples, such calibration may also, or alternatively, occur while content is being projected onto the mechanically animated surface 107 of the animatronic 106. In such examples, the initial movement of the animatronic 106 may be determined by the performance system 102 based on the initial content that has been or will be projected onto the mechanically animated surface 107 of the animatronic 106.

[0059] In block 302, the performance system 102 determines the orientation of the mechanically animated surface 107 relative to one or more projectors 104. The orientation may be determined using various sensors 108 associated with the animatronics 106 and / or the environment of the animatronics 106. In various examples, the sensors 108 may include visual sensors that obtain visual feedback regarding the positions of the animatronics 106 and the projector 104. For example, during a calibration step, the projector 104 may project a grid, reference points, or other calibration image onto the mechanically animated surface 107. A visual sensor (e.g., a camera) may then use the location of the grid or other reference points on the mechanically animated surface 107 to obtain position information for the mechanically animated surface 107 and the projector 104 relative to the mechanically animated surface 107. In some examples, sensors 108 may further include sensors that obtain position data for motors of animatronics 106. For example, position information for motors 114 of animatronics may be received from mechanical control system 112 and used to assist in determining the position of some or all of mechanically animated surface 107. Such additional position data may be used in addition to position data from the projection to determine the orientation of projector 104 relative to mechanically animated surface 107.

[0060] The position data of the mechanically animated surface 107 and the position data of the projector 104 are provided to the performance system 102 to determine, in general, an orientation of the projector 104 relative to the mechanically animated surface 107. In various examples, the orientation may include an orientation of a known point of the projector 104 relative to various markers on the mechanically animated surface 107. In general, the orientation of the projector 104 relative to the mechanically animated surface 107 is provided to the rendering engine 110 to render content based on one or more of those orientations.

[0061] In some examples, when multiple projectors are used to project content 105 onto mechanically animated surface 107, performance system 102 may determine the orientation of each projector relative to mechanically animated surface 107, and in some examples, the orientation of the projectors relative to each other. Similarly, when projector 104 (or multiple projectors) projects content onto multiple mechanically animated surfaces, performance system 102 may determine the orientation of each mechanically animated surface relative to its associated projector or projectors.

[0062] The performance system 102 determines parameters based on the orientation of the mechanically animated surface 107 at block 304. These parameters may be utilized by the rendering engine 110 to render content to be presented using the projector 104 and the mechanically animated surface 107. In various examples, the parameters may be characteristics of the projector 104 and / or the projection 105 that may be affected by the orientation between the projector 104 and the mechanically animated surface 107. In various examples, the parameters may include the projection angle, the projection ratio, and the projection brightness.

[0063] The performance system 102 determines such parameters based, in general, on the orientation of the mechanically animated surface 107 relative to the projector 104. For example, the rendering engine 110 may determine the correct angle of projection so that when the content 105 is projected onto the mechanically animated surface 107, each element of the content 105 is properly aligned with each feature of the mechanically animated surface 107. The rendering engine 110 may similarly determine the ratio of the projected content 105 so that the content 105 is properly scaled when projected onto the mechanically animated surface 107. The performance system 102 may also determine the brightness of the projected content 105 based on the orientation of the mechanically animated surface 107 relative to the projector 104. For example, to maintain the appearance (e.g., color tone, opacity, etc.) of the content 105 when projected onto the mechanically animated surface 107, a higher brightness may be used the further the mechanically animated surface 107 is from the projector.

[0064] In some examples, the rendering engine 110 may utilize the AI ​​model 128 to determine the parameters. For example, the rendering engine 110 may provide orientation information and / or input from the sensors 108 to the AI ​​model 128, and the AI ​​model 128 may return a set of parameters based on the orientation information. In some examples, the rendering engine 110 may provide input from additional sensors along with location information to the AI ​​model 128. For example, the rendering engine 110 may provide the orientation of the projector 104 relative to the mechanically animated surface 107 to the AI ​​model 128 along with input from sensors that detect ambient light in the environment of the animatronic 106. Thus, the AI ​​model 128 can generate a set of parameters that consider both the orientation of the projector 104 relative to the mechanically animated surface 107 and the ambient light level around the animatronic 106.

[0065] At block 306, the performance system 102 dynamically renders the content 105 based on the parameters. The rendering engine 110 generally renders the content 105 based on the determined parameters. Dynamic rendering of the content includes rendering frames of the content 105 in real time or near real time, which allows the performance system 102 to adjust the display of the content 105 and / or the content 105 itself based on conditions of the projector 104, conditions of the animatronics 106, conditions of the environment of the projector 104 or the animatronics 106, etc. In some examples, the rendering engine 110 may further determine adjustments to the angle or other settings of the projector 104, the movement of the mechanically animated surface 107, etc. Furthermore, when multiple projectors are used to project the content 105 onto the mechanically animated surface 107, the rendering engine can render the content 105 such that the content 105 blends properly across the multiple projectors.

[0066] The performance system 102 presents the content 105 on the mechanically animated surface 107 at block 308. Broadly speaking, the projector 104 projects the content 105 onto the mechanically animated surface 107, and such content 105 is synchronized with the movement of the mechanically animated surface 107. After the content 105 is dynamically rendered at block 406, the rendering engine 110 communicates with the projector 104 to provide the content 105 to the projector 104 for projection onto the mechanically animated surface 107. In some examples, the rendering engine 110 may provide the content 105 to the projector 104 by continuously streaming subsequent frames of the content 105 to the projector 104 for presentation. Additionally or alternatively, the rendering engine 110 may provide larger blocks or portions of the content 105 to the projector 104 for presentation.

[0067] The rendering engine 110 may further communicate movement of the mechanically animated surface 107 to a mechanical control system 112 of the performance system. Such movement, in various examples, may include actuation of motors 114 of the animatronics 106, which actuate the mechanically animated surface 107. Such movement may be generated based on animation data of the characters of the animatronics 106. Such movement may also include timing of the actuation (e.g., via timestamps synchronized with the projection of the content 105) to synchronize the movement with the projection of the content 105 onto the mechanically animated surface 107. Broadly speaking, the mechanical control system 112 may translate such movement into individual actuation commands for individual ones of the motors 114.

[0068] 6 illustrates a process 400 for adjusting the projection of content onto a mechanically animated surface 107 based on sensor input. Process 400 may generally enable the performance system 102 to dynamically and automatically adjust the content 105 in response to changes in the animatronic 106's environment (e.g., changes in light or volume). Such adjustments may help maintain high fidelity of the animatronic 106 when the content 105 is projected onto the animatronic 106. Process 400 may also be utilized to achieve high-fidelity interactive animatronics. For example, the content 105 may be adjusted or dynamically rendered in response to sounds, movements, or other conditions detected in the animatronic 106's environment.

[0069] In some examples, process 400 may also be utilized to dynamically respond to changes in the orientation of the mechanically animated surface 107 relative to the projector 104. For example, the orientation of the mechanically animated surface 107 relative to the projector 104 may change when content 105 is being projected onto the mechanically animated surface 107 due to movement of the projector 104 or the animatronic 106, a guest accidentally colliding with the animatronic 106, etc. Using process 400, the dynamic rendering engine 110 may adjust the content 105 to dynamically respond to such changes, such that the projector 104 and the animatronic 106 do not need to be recalibrated in response to changes in orientation, thereby allowing the animatronic 106 to operate continuously.

[0070] At block 402, the performance system 102 receives input from the animatronic sensor 108. The sensor input may generally be received by the performance system 102 (e.g., the rendering engine 110) from the animatronic sensor 108. In various examples, such sensor input may include acoustic input, visual input, and / or tactile input (e.g., via a touchscreen, buttons, etc.). For example, the animatronic sensor 108 may include one or more microphones that receive acoustic input (e.g., to measure noise levels or pick up sounds in the environment of the animatronic 106). In some examples, the animatronic sensor 108 may include a light sensor that provides input regarding the amount of light in the environment of the animatronic 106. If the animatronic sensor 108 is a camera, the rendering engine 110 may receive a video stream, still images, or other input from the camera.

[0071] The rendering engine 110 may continuously receive input from the animatronic sensors 108. In such examples, the rendering engine 110 may monitor such input for changes from a previous or baseline sensor input. For example, input from a camera may be used to determine the orientation of the mechanically animated surface 107 relative to the projector 104 and may be monitored for changes in that orientation. In some examples, the rendering engine 110 or other components of the performance system 102 may continuously receive visual or audio input and monitor such input for attempts to interact with the animatronic 106. For example, the performance system 102 may analyze video input to detect people in the frame, predefined gestures in the frame, etc. Similarly, the performance system 102 may analyze audio input for sounds other than background noise to detect commands, questions, or other input from a user attempting to interact with the animatronic 106.

[0072] The performance system 102 derives parameters related to the content 105 from the sensor inputs at block 404. In various examples, such parameters may be derived from the orientation of the mechanically animated surface 107 relative to the projector 104, which orientation is derived from the sensor inputs. Additionally or alternatively, parameters may be derived from other types of animatronic sensors 108. For example, a light sensor may be used to derive a parameter related to the brightness of one or more projections.

[0073] In various examples, parameters may further be derived using inputs from the AI ​​model 128. In some examples, the performance system 102 may provide a single sensor input to the AI ​​model 128, and the AI ​​model 128 may return a corresponding parameter or set of parameters based on the sensor input. For example, the performance system 102 may provide an ambient light measurement to the AI ​​model 128, and the AI ​​model 128 may return a brightness parameter for the projected content 105. In some examples, the performance system 102 may provide multiple sensor inputs to the AI ​​model 128, and the AI ​​model 128 may return a set of parameters based on the inputs. For example, the performance system 102 may provide an ambient light measurement and an orientation of the projector 104 relative to the mechanically animated surface 107. The AI ​​model 128 may use both inputs to return a brightness parameter and a projection ratio parameter. Such parameters may be more accurate if the AI ​​model 128 has been trained with sensor fusion 126. For example, a brightness parameter that takes into account both ambient light and the orientation of the projector 104 relative to the mechanically animated surface may be more accurate than if the brightness parameter were based solely on ambient light measurements.

[0074] In some examples, the parameter may be an interactive parameter. For example, the interactive parameter may be an identification of a voice, a tactile input, a gesture, etc. In some examples, such a parameter may be an identified voice (e.g., a spoken word by a user interacting with the animatronic 106), an identified gesture (e.g., a "thumbs up" gesture identified in a video input), etc. Such a parameter may, in various examples, be identified using the AI ​​model 128. For example, the AI ​​model 128 or another AI model in communication with the performance system 102 may be configured to process audio input, visual input, or other types of input to identify a voice, a gesture, etc. Some such parameters may be derived using a predefined input. For example, if the input is a user selecting a particular button, the corresponding parameter may be the options associated with the button selection.

[0075] At block 406, the performance system 102 dynamically renders the next frame of content based on the parameters. In some examples, the dynamic rendering of the next frame of content may be performed by adjusting the next frame of content 105 already projected by the projector 104 based on the environmental parameters of the animatronics 106 to improve the fidelity of the content 105. For example, the rendering engine 110 may render the next frame of content 105 based on brightness parameters derived from one or more sensor inputs.

[0076] In some examples, the next frame of content may be newly rendered and / or selected based on the interactive parameters. In such examples, real-time puppet 130 may receive one or more interactive parameters and select the next frame of content. The next frame of content may be provided to rendering engine 110, rendered, and transmitted to projector 104.

[0077] In some examples, the performance system 102 may also dynamically render or determine the movement of the animatronics 106 so that the movement of the mechanically animated surface 107 is synchronized with the content 105 projected onto the mechanically animated surface 107.

[0078] The performance system 102, at block 408, provides the next frame of content to the projector 104, which is projecting the content onto the mechanically animated surface 107. Broadly speaking, the rendering engine 110 communicates with the projector 104 to provide the next frame of content 105 to the projector 104 for projection onto the mechanically animated surface 107. The rendering engine 110 may further communicate movement of the mechanically animated surface 107 to the mechanical control system 112. Such movement, in various examples, may include actuation of motors 114 of the animatronics 106, which cause the mechanically animated surface 107 to move.

[0079] The descriptions of some embodiments contained herein are merely exemplary in nature and in no way intend to limit the scope of the disclosure or its application or uses. In the detailed description of the embodiments of the systems and methods of the present invention described above, reference is made to the accompanying drawings, which form a part of this specification and which show specific examples of embodiments in which the described systems and methods may be practiced. These embodiments have been described in sufficient detail to enable those skilled in the art to practice the systems and methods of the present disclosure; it will be understood that other embodiments may be utilized, and structural and logical changes may be made without departing from the spirit and scope of the present disclosure. Moreover, for purposes of clarity, detailed descriptions of some features have not been provided if they would be apparent to those skilled in the art. This is to avoid obscuring the description of the embodiments of the present disclosure. Therefore, the detailed descriptions are not to be construed in a limiting sense, and the scope of the present disclosure is defined only by the appended claims.

[0080] From the foregoing, it will be appreciated that, although specific embodiments of the invention have been described herein for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the invention.

[0081] The details set forth herein are by way of example and are presented solely for the purpose of illustratively describing preferred embodiments of the invention, and are provided to provide what is believed to be the most useful and most easily understood explanation of the principles and conceptual aspects of various embodiments of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention, and the description is provided by way of figures and / or examples that will make apparent to those skilled in the art how some forms of the invention may be implemented in practice.

[0082] As used herein, unless otherwise indicated, the words "a" and "an" are to be construed to mean "one," "at least one," or "one or more." Unless otherwise required by context, singular terms as used herein include pluralities and plural terms include the singular.

[0083] Unless the context clearly dictates otherwise, throughout the specification and claims, words like "comprise" and "comprising" should be construed in an inclusive sense (i.e., "including, but not limited to") and not in an exclusive or exhaustive sense. Words using the singular include the plural and vice versa. Furthermore, the words "herein," "above," and "below," and equivalent terms, as used in this application, refer to this application as a whole and not to any particular portions of this application.

[0084] Of course, it will be understood that any example, embodiment, or process described herein may be combined with one or more other examples, embodiments, and / or processes, or may be separated and / or implemented across separate devices or device portions in accordance with the systems, devices, and methods of the present invention.

[0085] Finally, the foregoing description is intended to be merely illustrative of the inventive system and should not be construed as limiting the appended claims to any particular embodiment or embodiments. Thus, while the inventive system has been described in particular detail with reference to exemplary embodiments, it will still be understood that various modifications and alternative embodiments may be devised by those skilled in the art without departing from the broader intended spirit and scope of the inventive system, as set forth in the claims that follow. Accordingly, the specification and drawings are to be taken as illustrative, and are not intended to limit the scope of the appended claims.

Claims

1. determining an orientation of a mechanically animated surface relative to one or more projectors configured to project content onto the mechanically animated surface; dynamically rendering the content based on one or more parameters derived from the orientation of the mechanically animated surface relative to the one or more projectors; presenting the content on the mechanically animated surface using the one or more projectors; A method comprising:

2. 10. The method of claim 1, further comprising dynamically rendering one or more movements of the mechanically animated surface, the movements of the mechanically animated surface being synchronized with the content.

3. 3. The method of claim 2, further comprising directly controlling one or more motors configured to actuate the mechanically animated surface to cause the mechanically animated surface to perform the one or more movements during the presentation of the content on the mechanically animated surface.

4. The method of claim 1 , wherein the content is dynamically rendered using vertex and animation data.

5. 10. The method of claim 1 , wherein the orientation of the mechanically animated surface relative to the one or more projectors is determined based on input from one or more animatronic sensors associated with the mechanically animated surface and the one or more projectors.

6. The method of claim 1 , wherein the one or more parameters include one or more of an angle of the projection, a ratio of the projection, and a brightness of the projection.

7. receiving input from at least one animatronic sensor associated with an animatronic, the animatronic including a mechanically animated surface; deriving position information for the mechanically animated surface from the input; dynamically rendering a next frame of content to be projected onto the mechanically animated surface based on the position information; providing the next frame of the content to at least one projector projecting the content onto the mechanically animated surface; A method comprising:

8. 8. The method of claim 7, further comprising dynamically rendering one or more movements of the mechanically animated surface based on the position information, the movements of the mechanically animated surface being synchronized with the next frame of the content.

9. 10. The method of claim 8, further comprising imparting the one or more movements of the mechanically animated surface to a control system of the animatronic.

10. The method of claim 7 , wherein the next frame of the content is dynamically rendered using vertex data and animation data.

11. The method of claim 7 , wherein the at least one animatronic sensor is a camera.

12. The method of claim 7 , further comprising receiving further input from the sensor while the content is being projected onto the mechanically animated surface.

13. The method of claim 7 , wherein the next frame of the content is dynamically rendered using a game engine.

14. one or more projectors; animatronics, including mechanically animated surfaces; a performance system in communication with the one or more projectors and the animatronics, the performance system configured to dynamically render a projection of content from the one or more projectors onto the mechanically animated surface and one or more movements of the mechanically animated surface, the projection synchronized with the one or more movements of the mechanically animated surface; A system including:

15. 15. The system of claim 14, wherein the performance system includes a real-time rendering engine in communication with the projector and with a mechanical control system of the animatronic.

16. The system of claim 14 , wherein the performance system is further configured to determine the one or more movements of the mechanically animated surface.

17. 15. The system of claim 14, wherein the performance system is further configured to determine an orientation of the mechanically animated surface relative to the one or more projectors, and wherein the performance system is configured to dynamically render the projection based on the orientation of the mechanically animated surface relative to the one or more projectors.

18. 15. The system of claim 14, wherein the step of rendering the content comprises rendering the content to be presented by two or more projectors.

19. 15. The system of claim 14, further comprising one or more animatronic sensors associated with the animatronic, wherein the performance system is further configured to derive positional information of the mechanically animated surface from input to the one or more animatronic sensors and dynamically render a next frame of the content based on the positional information.

20. 20. The system of claim 19, wherein the performance system is further configured to provide the next frame of the content to the one or more projectors.

Images