System and method for media projection of animated figures

Abstract

A method of generating dynamic media for representation by projection onto an animated figure includes defining a computer-generated model of the animated figure via a processing circuit, operating a manufacturing system via the processing circuit to generate a tangible model based on the computer-generated model, generating a modified computer-generated model via the processing circuit based on the tangible model, simulating the projection of an image onto the modified computer-generated model via the processing circuit, and operating a projector via the processing circuit to project an image onto the tangible model based on the projection of the simulated image onto the modified computer-generated model.

Description

[Technical field]

[0001] (Cross-reference of related applications) This application claims priority to and the benefit of U.S. Application No. 63 / 212,375 (Systems and Methods for Animated Figure Media Projection), filed June 18, 2021, which is incorporated herein by reference for all purposes. [Background technology]

[0002] This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present technique, which are described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. As such, it should be understood that these statements are to be read in this capacity, and not as admissions of prior art.

[0003] Amusement parks and other entertainment venues include animated figures (e.g., robotic characters) to entertain guests while they are waiting in line for ride attractions or during the ride experience, among other attractions. Certain animated figures can be brought to life by projection mapping, which traditionally directs a pre-determined appearance at the animated figure. For example, certain animated figures may be visually supplemented with a set of pre-recorded or fixed images that can be aligned with the pre-programmed movements of the animated figure. While such techniques may provide more entertainment than a flat display surface, developments are now recognized that may further immerse guests in a particular attraction, ride, or interactive experience. For example, certain animated figures have internally located projectors that generate unrealistic background lighting, or that cast light via interior or rear projection through the translucent projection surface of the animated figure. Thus, it is now recognized that it is desirable to provide animated figures with the ability to contextually blend with the environment in a realistic and convincing manner, as well as to make the animated figures appear more lifelike.

[0004] (summary) Certain embodiments commensurate in scope with the initially claimed subject matter are summarized below. These embodiments are not intended to limit the scope of the present disclosure, and these embodiments are intended only to provide a brief summary of certain disclosed embodiments. Moreover, the present disclosure encompasses a variety of forms that may be similar to or different from the embodiments set forth below.

[0005] In one embodiment, dynamic media generation for presentation by projection onto an animated figure includes defining a computer-generated model of the animated figure by a processing circuit, operating a manufacturing system to generate a tangible model by the processing circuit based on the computer-generated model, generating a modified computer-generated model by the processing circuit based on the tangible model, simulating, by the processing circuit, the projection of an image onto the received computer-generated model, and operating, by the processing circuit, a projector to project an image onto the tangible model based on the simulated projection of the image onto the modified computer-generated model.

[0006] In one embodiment, an amusement park attraction system includes a projector configured to output an image; and a control system configured to operate the projector to define a first computer generated model corresponding to the animated figure, simulate the projection of the image onto the first computer generated model to determine first simulated projection data, transmit instructions to generate a first tangible model based on the first computer generated model and the first simulated projection data, update the first computer generated model to obtain a second computer generated model based on an electronic scan of the first tangible model, simulate the projection of the image onto a second computer generated model to determine second simulated projection data, transmit instructions to generate a second tangible model based on the second computer generated model and the second simulated projection data, and output the image to the second tangible model.

[0007] In one embodiment, one or more tangible, non-transitory, computer-readable media contain instructions that, when executed by at least one processor, cause the at least one processor to operate a manufacturing system to manufacture a tangible model of the animation figure, generate a computer-generated model of the animation figure based on an electronic scan of the tangible model, cause a first movement of the computer-generated model, cause a second movement of the animation figure based on the first movement of the computer-generated model, and operate a projector to output an image to the animation figure based on the second movement of the animation figure.

[0008] These and other features, aspects and advantages of the present disclosure will be better understood when the following detailed description is read in conjunction with the accompanying drawings, in which like parts represent like elements throughout the drawings. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic diagram of a system for providing dynamic media in accordance with an aspect of the present disclosure. [Diagram 2] 1 is a block diagram of a system for providing dynamic media and corresponding system actions according to an aspect of the present disclosure. [Diagram 3] 3 is a process flow diagram illustrating a process for implementing the character assets step of the process action of FIG. 2 according to an aspect of the present disclosure. [Figure 4] 3 is a process flow diagram illustrating a process for performing a direction analysis step of the process action of FIG. 2 according to an aspect of the present disclosure. [Diagram 5] FIG. 3 is a process flow diagram illustrating the process of the animation / rigging step of the process action of FIG. 2 according to an aspect of the present disclosure. [Figure 6] FIG. 3 is a process flow diagram illustrating a process for implementing a verification step of the process action of FIG. 2 according to an aspect of the present disclosure. [Figure 7]FIG. 3 is a process flow diagram illustrating a process for implementing the tangible design step of the process action of FIG. 2 according to an aspect of the present disclosure. [Figure 8] FIG. 3 is a process flow diagram illustrating a process for performing the tangible fabrication / introduction step of the process action of FIG. 2 in accordance with an aspect of the present disclosure. [Figure 9] FIG. 3 is a process flow diagram illustrating a process for performing the character performance step of the process action of FIG. 2 according to an aspect of the present disclosure. [Figure 10] FIG. 3 is a process flow diagram illustrating a process for performing the integration steps of the process actions of FIG. 2 in accordance with an aspect of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] One or more specific embodiments of the present disclosure are described below. In an effort to provide a concise description of these embodiments, not all features of an actual implementation are described herein. It should be recognized that in the development of any such actual implementation, as in any optical or design project, numerous implementation-specific decisions must be made to achieve the developer's particular goals, such as complying with system-related and business-related constraints that may differ from one implementation to another. Moreover, such development efforts tend to be complex and time-consuming, but may be a routine exercise for those skilled in the art in designing, fabricating, and manufacturing.

[0011] When introducing elements of various embodiments of the invention, the articles "a," "an," and "the" are intended to mean that there are one or more of the element. The terms "comprising," "including," and "having" are inclusive and mean that there may be additional elements other than the listed elements. In addition, it should be understood that references to "one embodiment," "one embodiment," or "some embodiments" of the invention are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the described features.

[0012] Theme parks and other such entertainment venues continue to grow in popularity. Additionally, there is a strong demand for immersive experiences within such entertainment venues. Thus, there is a currently recognized need to improve thematic effects to provide a more immersive impact on guests and a realistic experience that truly magically immerses them in the themed environment. One of the more challenging aspects of providing such immersion involves creating animated figures (often used for themed entertainment) that appear more lifelike than traditional approaches can achieve. Unrealistic animated figures can cause a breakdown in immersion. In particular, providing a character (e.g., human) face for an animated figure is recognized as particularly difficult as visitors must adapt to highly detailed aspects of the facial structure and interactions that are subtly perceived as unrealistic. Problems identified with traditional approaches that prevent the suspension of disbelief include the insufficient illusion of life provided only by three-dimensional (3D) forms (e.g., robotics), artificial skin actuation that only shows a portion of the realistic facial expression, projections that can be easily noticed based on light emission from within the animated figure, etc. Thus, embodiments are directed to control modalities that coordinate the pre-processing of projected media, the external projection onto tangible mechanical assets (e.g., animated robotic figures), and the layering of effects from these actions to provide a realistic experience. As described below, certain modalities and procedures are implemented based on centralized or decentralized processor-based control along with steps involving physical transitions and actions based on automation and / or human interaction.

[0013] In view of the foregoing, the present embodiments are generally directed to methods and systems for creating media projections onto animated figures in amusement attractions. In particular, the present embodiments are directed to providing attraction features that facilitate immersion in a theme by providing realistic, lifelike, or other immersive media projections onto the exterior of an animated figure in coordination with other controlled aspects of the animated figure. Layered interactions (e.g., projected animations in coordination with fabricated 3D projection surfaces and the motion of the animated figure) provide for the projected media, control features, sounds, and aspects of the animated figure. Such layered interactions or behaviors provide a highly realistic and nuanced presentation that can be noticed by an observer. Such layering is achieved in part based on utilizing several interaction data sources or data streams (e.g., the current state of the animated figure, the quality of guest actions or passives, the actions of performers behind the scenes or on stage) to generate detailed, complex, and subtle behaviors that correspond to reality. Due to the complex nature of such interactions, in some embodiments, control commands from an artificial intelligence (AI) or machine learning engine can adjust media, projection position, animation speed, projection surface location, etc. to create a realistic and immersive experience. Additionally, routines of mass and variable repetition are included to incrementally improve the production site, and are repeated until a recognized level of realism or immersion is achieved.

[0014] In particular, the techniques disclosed herein provide the equipment and basis for presenting what may be referred to herein as dynamic or active media. Dynamic media is defined herein as media that layer the provision of a dynamic and immersive experience to guests in which the animated figure more closely resembles a real person or character than is traditionally realized. In particular, providing dynamic media as described herein includes projection mapping images onto the exterior surface of the animated figure in conjunction with preparing / actuating the components of the animated figure and based on various inputs, thereby providing a more lifelike animated figure. Projection mapping may be particularly beneficial when compared to certain animated figure systems that project internally through semi-transparent surfaces of the animated figure, creating a resultant unnatural background lighting or ethereal glow. Furthermore, dynamic media such as the projected images and / or movement of the animated figure may be interactive or responsive, such as based on interaction with the guest or other sensed parameters. Thus, instead of providing a pre-determined show effect, the show effect may be dynamically generated, and may vary, such as for different action cycles, to provide a more immersive experience to guests.

[0015] As described herein, the present embodiments and techniques define a foundation that allows for the preparation of dynamic media using character definition, direction analysis, animation rigging, projection control, animation figure control, animation figure actuation, and the like. As described by this disclosure, these aspects are integrated or layered to provide an overall effect of optical, physical, or audio interaction based on a large amount of input and control (e.g., show control, controlling various multimedia controllers). This overall effect is the basis for providing dynamic media and related immersive experiences to the viewer. For example, the results of the techniques described herein may provide a computer-generated model that more closely and accurately corresponds to a physical, tangible, real-life animated figure, and the computer-generated model may be controlled (e.g., to impart motion to the computer-generated model) to facilitate corresponding control of the physical animated figure, and to facilitate more responsive control of the physical animated figure and the image show effects projected thereon.

[0016] While the examples provided herein generally focus on mimicking the head or face of a character (e.g., a human character), it should be understood that the present embodiment is not directed to that example. Furthermore, any manner of animated character may be represented by the present embodiment. As an example, features not typically considered to be facial may be utilized and may be referred to herein as facial (e.g., the front of a car, or a factory may have characteristics that enhance or add to the impression of a human face). Some aspects or embodiments may still be perceived as mimicking real facial features, such as those of human characters, which are perceived as challenging in the relevant areas due to the viewer's habituation and adaptation to such features. Moreover, as used herein, animated figures may include any suitable physical object or prop that can move within the space occupied by the object. Additionally, the animated figures may include clothing (e.g., a shirt worn by a user), deformable objects (e.g., a balloon that may inflate and / or deflate), a ball that may be rolled and / or thrown, or any other suitable object capable of movement, and the image projected onto the object may be dynamically adjusted based on such movement of the object. Note that although the present disclosure is primarily described in the context of an amusement park, the techniques and embodiments described herein may be applied to medical applications, transportation applications, architectural applications, or any other applications in which computer-generated models and corresponding physical models or figures may be utilized.

[0017] Procedures according to the present disclosure for dynamic media (applicable to the procedures illustrated in FIGS. 2-10) include a variety of different steps and procedural aspects. Some of these steps or procedures may be performed in parallel or in different orders. Some steps may be processor-based actions and may involve controlled equipment (e.g., actuators or 3D printers). Additionally, some procedures may be performed iteratively to achieve a desired outcome. Thus, while the steps of various different procedures may be described in a particular order herein, the steps of the procedures may not have to be performed in the order presented as described in the present disclosure. While some particular steps of the actions may necessarily occur before other particular steps (e.g., as dictated by logic), the recitation of a certain order of actions is provided primarily for ease of exposition. For example, indicating that a first or initiating step includes a particular action is not intended to limit the scope of the present disclosure to such first step. Rather, it should be understood that additional steps may be performed, certain steps may be omitted, recited steps may be performed in alternate orders or in parallel, etc., as appropriate. However, the disclosed order of actions may be limited when so presented.

[0018] FIG. 1 depicts a system 10 operable to create and provide dynamic or active media in accordance with the present disclosure. System 10 may include a control system 11, which may include control circuitry (e.g., one or more automated controllers, one or more electronic controllers, one or more programmable controllers) configured to implement various techniques described herein to provide dynamic media. Control system 11 may include one or more processors or processing circuitry 14, and a tangible computer-readable medium 18. Tangible computer-readable medium 18 may include volatile memory, such as random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM), optical devices, hard disk devices, solid-state drives, or any other suitable non-transitory computer-readable medium in which instructions may be stored. Processor 14 may be configured to execute instructions. For example, processor 14 may include one or more application specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), one or more general-purpose processors, or any combination thereof.

[0019] The control system 11 can be configured to operate the dynamic media to provide a desired experience to provide a show effect to the guests. For example, the system 10 may include a scene or stage area 1 in which the projector 2, the animated figure 3, and other props 4 may be positioned. The control system 11 can be configured to operate the projector 2 to output content or images to the animated figure 3 to provide a desired (e.g., realistic) appearance of the animated figure 3. The control system 11 can also be configured to cause movement of the animated figure 3, such as through the action of the actuators 5, to steer the movement of the animated figure 3. Additionally, the control system 11 may adjust the image output according to the movement of the animated figure 3 relative to the projector 2 and each other, and coordinate the action between the projector 2 and the animated figure 3 to provide a realistic appearance or effect that may not otherwise be easily achievable. The control system 11 can also be configured to operate other props 4, such as light emitters, or sound emitters, fog / smoke effect systems, etc., to complete or complement the effects provided via the action of the projector 2 and / or the animated figure 3. In some embodiments, the control system 11 may be communicatively connected to sensors 21 configured to determine various operating parameters, and the control system 11 may be configured to act based on the operating parameters. For example, the operating parameters may include the position, orientation, etc. of the animated figures 3 and / or projectors 2 in the scene area, audio output in the scene area (e.g., by other props 4), and / or the appearance of an image projected onto the animated figures 3.

[0020] Guests may be located in guest area 6 of system 10 and may be able to view effects provided by scene area 1. By way of example, scene area 1 and guest area 6 may be part of an amusement park attraction system, which may operate to provide a unique experience to guests. Guest area 6 may include a vehicle that may move (e.g., along a track or walkway) relative to scene area 1, and guest passengers on the ride may view scene area 1 while the ride passes through scene area 1. Additionally or alternatively, guest area 6 may be fixed relative to scene area 1, such as for a dramatic performance, and may include a grandstand type arrangement. In any case, interaction of scene area 1 through control system 11 may provide dynamic media to entertain guests in guest area 6.

[0021] The control system 11 may also be configured to iteratively perform various actions to generate desired dynamic media, such as facilitating the creation, fabrication, interaction, installation, etc. of the projector 2, the animated figure 3, and / or the props 4. For example, the control system 11 may be configured to acquire desired character assets to generate a model representing the animated figure 3. In one embodiment, the control system 11 may be configured to operate a scanning system 7, such as a 3D scanning device, to capture an image of the real-world object 8 and create a computer-generated model 9 based on the image. The control system 11 may receive the computer-generated model 9 from the scanning system 7. The computer-generated model 9 may be used to aid in the design and creation of the animated figure 3. For example, the control system 11 may include a display 13 that a user utilizes to view the computer-generated model 9, and the control system 11 may include a user interface (UI) 15, such as a mouse, keyboard, touch screen, track pad, dials, buttons, etc., that a user may utilize to adjust various features of the computer-generated model 9.

[0022] The control system 11 may also be configured to operate a manufacturing system 17, such as a 3D printing machine and / or a modeling machine (e.g., an injection molding machine), to create a tangible model 19. The tangible model 19 may be a more simplified version of the animation figure 3 (e.g., a portion of the animation figure, a miniature model of the animation figure) for purposes of initial evaluation and / or testing. As an example, an initial decision may be made as to whether an image output onto the tangible model 19 (e.g., via the projector 2) will look desirable before beginning a more complex design and / or manufacturing process of the animation figure 3. In some embodiments, the computer-generated model 9 may be adjusted based on the appearance of the tangible model 19 (e.g., an image projected onto the tangible model 19). For example, the control system 11 may operate a scanning system 7 to create a computer-generated model 9 based on the tangible model 19 (e.g., an image captured of the tangible model 19). Thus, the computer-generated model 9 may be updated to more closely represent the tangible model 19 instead of the real-world object 8. Such a computer generated model 9 can also be adjusted via the control system 11 and subsequently used to create a subsequent physical model 19 via the manufacturing system 17 .

[0023] In this manner, numerous iterations of the computer-generated model 9 and / or the tangible model 19 may be created until a representation of the desired appearance of the animated figure 3 is produced along with the imagery projected thereon. When a satisfactory appearance (e.g., of the tangible model) is achieved, the control system 11 may facilitate production of the animated figure 3. For example, the control system 11 may complete the computer-generated model 9, such as by creating a design that is more suitable for mechanical manufacturing processes. The final production of the animated figure 3 may then be produced based on the completed computer-generated model 9 and may be prepared for introduction into the scene area 1. Actions of the animated figure 3, such as movement as supported by actuators 5, may be based on the computer model 9 (e.g., movement simulated via the computer-generated model 9).

[0024] 2 depicts a system 10 capable of implementing a process 12 for providing dynamic media in accordance with the present disclosure. Process 12 may be representative of various computer actions performed via a processor 14 based on instructions 16 stored on a tangible computer-readable medium, including actions using AI and machine learning. Some actions may not be directly controlled by the processor (e.g., certain physical actions). However, even certain physical actions, such as manufacturing (e.g., by a 3D printer), involve some aspect of control. Each of the steps of process 12 is discussed in more detail below.

[0025] The start of the process 12 is represented by a start block 20 indicating the beginning of a character assets step 24, which may provide characters or elements on which the projected media may be based, and a direction analysis step 26, which may provide scenes, venues, stories, environments, settings and / or functions on which the animated figure may perform. The character assets step 24 and direction analysis may be performed in parallel or in succession. The results from these steps are then utilized in an animation / rigging step 28, which may generate a computer-generated model representing the animated figure on which the media will be projected. A subsequent animation / rigging step 28, a validation step 30 may be performed to complete the computer-generated model of the animated figure, and a tangible design step 32 may then be performed to prepare the animated figure for production, such as to provide mechanical digital assets based on the completed computer-generated model. The animation / rigging step 28, validation step 30 and tangible design step 32 include an iterative manner to refine the results to a desired level of audience-perceived realism. Once the tangible design action 32 is specified as complete, a tangible fabrication / introduction step 34 is performed to create the animated figure and / or a stage or environment in which the animated figure may be implemented to provide the dynamic media. A character performance step 36 to determine the movement of the animated figure and / or the projection of images onto the animated figure is then performed to adjust the movement of the animated figure and the dynamic media action followed by an integration step 38 to introduce the dynamic media into the scene area and complete the dynamic media action. Once the integration step 38 is completed, the overall process 12 may be considered to be over with the completed dynamic media generated and created as the product of the process 12, as an end block 40 is specified. The resulting dynamic media may then be operated in an attraction system of an amusement park system, or the like, and presented to guests.For example, completion of integration step 30 may provide a computer-generated model and a physical animation figure that closely corresponds to the computer-generated model. The dynamic media effect may include using the computer-generated model to simulate movement of the computer-generated model and / or images projected onto the computer-generated model, and providing a corresponding movement of the physical animation figure and / or images projected onto the physical animation figure. Details relating to individual aspects of the overall process 12 are discussed in more detail below.

[0026] FIG. 3 is a process flow diagram illustrating a process 50 for implementing the character assets step 24 of the process 12 according to the present disclosure. The character assets step 24, sometimes referred to as character asset creation, involves the development or selection of a character asset to be prepared for projection media (e.g., projection media for representing human facial animation) that displays the projection of the character or aspects of the character as a component of providing dynamic media. The process 50 begins as indicated by start block 52 and proceeds initially to determining a character, as indicated by block 54. Determining a character may include selecting a particular subject (e.g., a person, a sculpted image, an animal). In selecting a character, a model or form (e.g., a 3D digital human facial model) of the character may be created as indicated by block 56. The form or model (e.g., a 3D model) may be referred to as a computer-generated model. For example, a predefined or predetermined model (e.g., a 3D model) or a computer-generated model may exist for the selected subject, and the predefined model may be selected or retrieved from storage. Additionally or alternatively, new computer-generated models may be defined for characters, such as for subjects that do not have an associated model and / or for subjects that may have outdated models. For example, 3D scanning techniques use 3D scanning technology (typically with very high-precision data capture) for the creation of 3D computer-generated models of real-world objects such as statues, objects, or creatures. In particular, the 3D scanner captures a large number of images of the object and generates an electronic scan (e.g., an electronic mapping of the subject). The captured images may be fused, stitched, combined, or otherwise integrated into a 3D computer-generated model that is an electronic 3D replica of the subject or object, including various points or parts of the subject mapped to a 3D coordinate system.Once a computer-generated model is defined (e.g., via scanning), related viewing and / or editing software applications may enable viewing and / or manipulating the computer-generated model on a display (e.g., a computer screen) from a variety of different angles. For example, the software application may facilitate virtually rotating the computer-generated model on a display (e.g., using a user interface such as a mouse or keyboard).

[0027] In some embodiments, the computer-generated model can be graphically modified, as represented by block 57. For example, the color, texture, features (e.g., facial features, hair, jewelry or accessories) of the computer-generated model can be added, removed, or otherwise adjusted. The model proportions or size of the computer-generated model may also be modified. In this manner, the computer-generated model can facilitate greater customization of the appearance of a selected character to achieve a desired portrayal, such as based on a projected media implementation.

[0028] As illustrated in FIG. 3, the process 50 proceeds to define the character performance as represented by block 58. Defining the character performance may be performed before, after, or in parallel with creating the computer-generated model (block 56) and / or modifying the computer-generated model (block 57). The step represented by block 58 may include capturing real-life video and other motion data of the actor's character performance, including associated facial feature movements (e.g., movements used by a subject while acting out a scene). Motion capture techniques may be used to capture such video images and motion data. This captured video-related data may then be used in conjunction with the computer-generated model to define the character performance. In particular, motion capture techniques and data point combinations may be used to define the character performance. For example, motion capture points on the actor's lips may be identified for later use with corresponding points (e.g., coordinate points) in the representation of the lips of the computer-generated model (e.g., electronic face model), and the associated motion data of the motion capture may be applied to the lips in the computer-generated model. In other words, the motion of the computer-generated model (e.g., in a 3D coordinate system) may be determined based on motion capture of the corresponding subject that the computer-generated model represents. As another example, 3D animation techniques may be used to define the character performance. In particular, software applications may be used to edit and manipulate the underlying images and / or the computer-generated model, e.g., to create a sequence of images that coordinate to provide an animation. Further manipulation of the computer-generated model may thus define the body torso information of the character. This aspect of the present embodiment allows for the use of numerous conventional animation techniques.The defined animation or movement of the character (e.g., the character's position over time) may be stored as animation data that may be readily retrieved for implementation and / or additional modification. Additionally, audio may be recorded as part of or separate from the visual portion of the character performance. The audio may be combined with the video data to provide a completed character performance. However, in some circumstances, the video data may be generated and / or used without the audio data.

[0029] Using the acquired data, the process 50 can generate or determine a digital asset (e.g., a fixed face model) that can be stored, retrieved, and / or further manipulated to perform different motions and / or provide different appearances, as represented by block 60. Data relating to the data, an arrangement of position points or configurations (e.g., facial expressions), etc., can be utilized to define a digital asset (e.g., a digital facial asset) that can represent the appearance of a character at a particular point in time. This step can include using a structural construction algorithm (e.g., a facial construction algorithm) that utilizes the available data to predict and adapt the appearance of certain types of structures (e.g., facial structures) to facilitate the portrayal of a wide range of motion-based actions (e.g., facial expressions). Furthermore, this procedure allows for the generation and simulation of various instances of expressions without requiring the specific capture of such motions in a previous step. Furthermore, as represented by block 62, a look development step can be performed on this digital asset to prepare associated graphics for use with a game engine, which can include software tools or application programming interfaces that facilitate building a graphics-based model of a subject or character. Specifically, using conventional techniques, this may include texturing, surface modeling, and assembly of geometry instances to facilitate animation that is realistic or otherwise immersive for use in providing dynamic media. At this point, the illustrated process 50 may be considered complete. However, as will be discussed below, the digital facial assets (e.g., as represented by block 28 in FIG. 1 and block 64 in FIG. 3) may subsequently be used in conjunction with animation and / or rigging to provide a completed computer graphic asset (e.g., after completion of look development).

[0030] 4 is a process flow diagram illustrating a process 70 for implementing the rendition analysis step 26 of process 12 in accordance with the present disclosure. Data obtained from process 70 may be used to facilitate layering of effects to provide dynamic media. For example, in accordance with an embodiment of the present disclosure, timing, position and orientation data obtained during process 70 may be utilized to control media displays, model actions and animated character actuation (e.g., robot actuation) in a concert. The start of process 70 is represented by start block 72, and as previously discussed, process 70 may be performed before, after or in parallel with character assets step 24.

[0031] As a first step in the process 70 of FIG. 4, a scene or venue specific rendition of the character is determined (step 74) to determine the implementation of the character in the scene area, which may include other show effects, props, or elements in addition to the character. The scene fixed rendition may be performed using computer modeling (including computer generated models of the scene area and animated figures), which may include mapping specific coordinates (including aspects of volumetric analysis) of the character in different poses or orientations in the 3D coordinate system of the scene area at different times. Additionally, as represented by block 76, line of sight and field of view may be determined or calculated for one or more guest views. Determining line of sight and field of view may include determining the mean or average view for a large number of different audience viewpoints, such as based on the alignment of the guest and / or guest area with respect to the real scene area (e.g., via computer generated models). For example, the movement or placement of the character in the scene area may be determined based on line of sight and field of view to provide visibility of the target portion of the character to the guest. Additionally, as represented by block 78, a duration of the scene is determined (e.g., the amount of time allotted or the amount of time generally used for the performance of a particular scene or expression). Determining the duration of the scene may include determination of time frames of aspects or portions of the scene, such as the total duration of running a single cycle of the scene, timestamps for when the scene begins and / or ends, etc. Determining the duration of the scene may additionally or alternatively include determinations related to characters in the scene, such as individual durations associated with maintaining various positioning of a character and thereby maintaining the resulting visibility of different portions of the character, time frames for performing movements by the character (e.g., transitioning between facial expressions), timestamps indicating the movement of the character relative to the movement of other elements of the scene (e.g., the character moves in conjunction with the movement of additional props), etc.

[0032] The process may also include an initial projection and tracking / camera design, which may include a virtual or physical placement of projectors and / or associated devices, such as devices (e.g., bogies), that may move the projectors, as represented by block 80. The initial projection and / or tracking design may simulate the appearance of an image output (e.g., from a guest's perspective) through the projector from a particular positioning (e.g., position, four directions) of the projector. Additionally, the process may include formal learning, as represented by block 82, based on cone / frustum projections, angles, field of view, guest line of sight, etc. Projection learning may be performed via software to help determine if the image projected by the projector to the character is distorted, obstructed (e.g., blocked or occluded by other props), or otherwise undesirable output. Thus, projection learning may be used to determine whether image data used to output the image should be adjusted, whether projector positioning should be changed, whether a different projector or projector type should be used, whether additional projectors should be incorporated, etc.

[0033] In block 84, adjustments to the projection adjustments based on character movement and configuration transitions may be provided by sensing the state (e.g., geometry, contours) of certain character configurations. For example, for characters with dynamic or active mouths (e.g., characters that are physically operable to transition mouth features between open and closed configurations), an open mouth configuration (e.g., a character's jaw is open) and / or a closed mouth configuration (e.g., a character's jaw is closed) may be determined. In other embodiments, a different configuration (e.g., a raised or lowered forehead configuration) may be determined. Additionally, a larger or fuller transition (e.g., morphing a ball into a star shape or a spiked ball) may be applied. Again, such data may be used to facilitate layering of effects that create realistic images by associating different projected images, types of projected images (e.g., images projected by a certain projector or from a certain direction), or categories of projected images (e.g., a happy expression of an open mouth configuration compared to a happy expression of a closed mouth configuration) with different possible configurations of a character. For example, an open mouth configuration of a physical character may be associated with a projection that aligns with an enlarged character's head configuration (relative to closed mouth geometry) to create a more realistic and immersive viewing experience. Additionally, it is believed that the results provided by the present embodiment should be advantageous over conventional representations in part due to the layering of such effects and the subtle details associated with such layering. Similar to process 50 depicted in FIG. 3, the data generated in process 70 may now be utilized in conjunction with animation and / or rigging (as represented in FIG. 1 by block 28 and in FIG. 4 by block 86) to provide a completed computer graphic asset.

[0034] FIG. 45 is a process flow diagram depicting a process 90 for implementing animation / rigging step 28 of process 12 according to the present disclosure. The animation / rigging step according to an embodiment of the present disclosure includes skeletal animation techniques (e.g., by a computer modeling program) for representing a 3D character model (e.g., a computer-generated model created in block 54 of process 50) using a series of virtually interlocking digital structural features, sometimes referred to as a digital skeleton. In particular, animation / rigging (or simply "rigging") may be used in this disclosure to refer to the process step of creating a digital skeleton structure of the 3D character model to define the interlocking manner and kinematic coupling of different parts of the 3D character model. This digital skeleton structure may be used to manipulate the 3D character model, which according to the present embodiment resembles a doll, for animation purposes. Such animation may include manipulation of the facial model and distortion of the facial model to produce different facial expressions.

[0035] Process 90 may begin at block 92, which may include preliminary rigging from the character assets (e.g., the digital facial assets created at block 60) assembled at process 50. That is, an initial digital skeletal structure of the character assets may be created to define an initial set of articulated aspects of the character assets. However, in some circumstances, the initial digital skeletal structure may be more complex, difficult, or excessively large than desired for a particular motion and / or projection mapping purpose associated with the character implementation. As shown in FIG. 5, the next step represented by block 94 may determine animation goal changes (e.g., rough animation), which may include applying a character performance (e.g., a character performance defined at block 58 of process 50, such as a video captured character performance) to a look development (e.g., a look development completed at block 62 of process 50). In this manner, the animation data created and stored as a result of block 58 may be modified (e.g., iteratively modified) based on the look development and other steps performed to enable the animation data to effect simulated motion of the digital skeletal structure. Process 90 may further include a structural analysis (e.g., facial analysis) based on the alignment of structural features (e.g., eyebrows, temples, cheeks, mouth, chin, lips) in a primary orientation or alignment (e.g., facial expression), as represented by block 96. Using data from this analysis, digital skeletons and digital joints (e.g., linkages between digital skeletons) may be determined in a primary orientation (e.g., primary facial expression). Thus, a particular subset of digital skeletons and digital joints may be determined that may be actuated or otherwise moved to form different primary orientations via animation data, such as to transform a 3D character model between different expressions of a corresponding fixed facial model for a scene.

[0036] Next, the 3D character model (e.g., a computer-generated model of a human face) is simplified (e.g., using a simplification algorithm) to make the 3D character model more amorphous in form while manipulating the main structures, as represented in block 98. Thus, certain digital skeletons and / or digital joints that may not be used to achieve different main orientations to the scene may be removed to reduce the amount of articulated set of aspects of the character assets. For example, the face model may be simplified (e.g., certain articulated facial structures, such as between the ears and jawbone, may be removed or disassociated from each other) and made more amorphous based on an algorithm designed to generate such simplifications exclusively for certain facial expressions. The simplification of the 3D character model may be performed manually (e.g., a user may manually select that digital skeletons and / or digital joints should be removed or retained) and / or automatically (e.g., based on automatic detection that the amount of use of certain digital skeletons and / or digital joints to provide main orientation falls below a threshold). The position and coupling between the digital skeleton and the digital joints remaining constant facilitates the identification and maintenance of key facial structures in such algorithms to achieve key orientations.

[0037] Once the simplified 3D character model is generated in block 98, the process 90 may proceed to an iterative portion of the process 90. This may include creating a computer graphic device and mechanical feature list for a particular actuation (e.g., facial features) as represented by block 100. That is, a list of different resulting outputs of the 3D character model's movement, etc. and / or appearance (e.g., facial expression) associated with actuating or moving different maintained digital skeletons and / or digital joints of the 3D character model (e.g., simplified character model) may be determined, such as to achieve the 3D character model's movement associated with the animation data created in block 58. Thus, the initially generated mechanical feature list may include many types of different outputs, such as more orientations than the primary orientation desired for the implementation. In addition, the iterative portion may include selecting, adjusting, modifying, or identifying a mechanical feature list, as represented by block 102. For example, a particular actuation of the 3D character model's digital skeleton and / or digital joints to be used to achieve the primary orientation via the animation data may be determined. In this manner, the initially generated mechanical feature list may be simplified to reduce the amount of mechanical features included in the mechanical feature list in order to focus on those desired to provide the primary direction. In some embodiments, animation data related to the movement of the 3D character model may be updated based on the mechanical feature list, such as to provide updated movement that may be more compatible or achievable via a digital skeletal structure (e.g., digital skeleton, digital joints) in view of the mechanical feature list.

[0038] Additionally, as represented by block 104, the iterative process may include projection simulation (via a processor-based computer simulator, such as computer graphic compositing and visual effects software, or other software) of the computer-generated imagery, which may take into account previously determined data including character appearance development, guest line of sight, movement, and / or primary orientation of the 3D character model. The projection simulation may generate representations of image data of the virtual content or images that, when projected onto the 3D character model, provide a realistic depiction of the appearance of the projection onto the physical real object representation of the 3D character model. As an example, for a projection simulation performed by a processor-controlled simulator (e.g., a simulation package or application running on a computer) and / or computer graphic compositing and visual effects software, the 3D character model (e.g., a facial 3D character model) may be positioned (e.g., rotated) relative to the virtual projector, and simulated lens qualities for the virtual projector may be virtually selected to match those of the actual projector. The simulation may then render the projection surface (e.g., structure and texture) onto the virtual projector and output the resulting image. If the projection surface and virtual projector sufficiently correspond to their real-world counterparts, the result can more accurately represent the appearance of the projected content in reality.

[0039] Continued iterations of these procedures depend on the output from decision block 106, which includes a decision as to whether the computer-generated image output from process 90 at this point is acceptable (e.g., based on user input or algorithm-based computer-implemented criteria). If the output is not acceptable (e.g., the predominant orientation caused by certain digital skeletal actuations and / or digital joints of the 3D character model is undesirable, the appearance of the image projected onto the 3D character model is undesirable, the coordination between the movement of the 3D character model and the adjusted image projected from the virtual projector is undesirable), the iteration may begin again at any one of blocks 100, 102, and / or 104. Selection of the appropriate procedure step for the continued iteration may depend on data related to the computer-generated image output at the relevant time. For example, a decision may be made as to whether the digital skeletal and / or joint actuations of the 3D character model based on a mechanical function list should be adjusted, whether the image output by the virtual projector should be adjusted (e.g., by adjusting image data to cause the virtual projector to output an image), or both based on the appearance and / or quality of the 3D character model along with the projected image. The assessment of the 3D character model and projected image may be based on algorithms, artificial intelligence, and / or user input, etc. Once the output computer-generated image is deemed satisfactory at block 106, a completed computer-generated asset is defined, which may include the computer-generated 3D character model, as represented by block 108. At this point, a change to the validation process occurs by block 110, as shown in FIG.

[0040] FIG. 6 is a process flow diagram illustrating a process 120 for implementing the validation process 30 of the process 12 according to the present disclosure. The validation process 120 may begin with the production of a tangible or physical version of the computer-generated model (e.g., 3D printing of the character's head and face), as represented by block 122. This may include automated 3D printing, AM techniques, modeling (e.g., injection molding), or component production (e.g., cutting) based on instructions from a controller after confirmation of a fully fabricated computer-generated image. The production of the tangible model may include refining actions to facilitate the use of the produced model for projection testing. For example, refining actions may include sanding, polishing, texturing, painting, trimming, punching, cutting, etc. Next, a projection test using multiple fixed poses of animation on the produced model may be performed, as represented by block 124. For example, the projection test may include projecting images of different facial expressions onto the manufactured facial structure model. To this end, as in block 104 of process 90, the image data used to simulate the image projection onto the computer-generated model and / or the image data adjusted based on the simulated projection onto the computer-generated model may be sent to a projector to output a tangible model image based on the image data. In some embodiments, various parts of the tangible model (e.g., a puppet rig) may be movable relative to one another to allow at least partial achievement of a primary orientation for the character represented by the tangible model. However, it should be noted that the tangible model created during the verification step 30 may be of a more simplified design (e.g., having a limited range of motion) with respect to a completed tangible animated figure for rendering and implementation in dynamic media.

[0041] Next, various iterative actions may be performed, as indicated by iteration and validation block 126. As indicated in FIG. 6, various steps may be included as iterative actions. In particular, in certain steps, physically or manually operable features may be incorporated, as indicated by block 128. For example, block 128 may represent the addition of flexible skin to the created head along with actuators (e.g., levers that can be operated by hand) to provide facial movement. In other embodiments, other features of the animated figure may be updated corresponding to the flexible skin (e.g., fur, scales, armor) and associated actuators. Another iterative step may include the incorporation of servo motors and controllers for automatic synchronized motion of associated actuations, as indicated by block 130. For example, actuators associated with facial features may be automatically operated (e.g., without the user manually operating each individual actuator) to display a range of expressions in the created head and face along with the flexible skin in various synchronized movements. It should be noted that the steps of blocks 128 and 130 may be mixed, eliminated, or kept separate depending on the aspects of the present disclosure.

[0042] Another iterative step may include block 132, which represents projection onto the tangible model assembled in blocks 128 and / or 120, to determine correspondence between mechanical features and projection characteristics of the tangible model (e.g., alignment of eye movements with mechanical positioning). In this regard, an iterative step as represented by block 134 may include scanning the tangible model (e.g., using a LiDAR (light detection and ranging) or 3D scanner) to determine an updated computer-generated model (e.g., electronic scanning or mapping) based on the tangible model. Previous procedures such as generating a skeletal structure, determining a mechanical feature list, and / or simulating a projected image may be performed using such updated or new computer-generated model. By way of example, the motion, positioning, and / or geometry of the tangible model may be obtained, such as via motion capture of the tangible model, and such motion, positioning, and / or geometry may be used to update animation data used to control the computer-generated model. In this manner, certain steps of the processes previously described, such as process 90 previously performed for the initially created computer-generated model, may be performed on the updated computer-generated model based on the tangible model. Such an act of initially providing or utilizing a more simplified tangible model may increase the efficiency of iteration and facilitate the transformation (e.g., via block 56 of process 50) from the initially created computer-generated model to a working physical structure without having to design and create multiple, more complex, completed animated figures.

[0043] Further, as illustrated in FIG. 6, data from previous steps in the iteration can be used to iteratively update and create the next tangible model and / or to determine whether to further iterate to provide an updated computer-generated model, such as by adjusting image data for projecting images onto the tangible model, or to complete the computer-generated model. In particular, as represented by block 136, a determination is made as to whether life-like functionality of the projection of the tangible model onto the form is observed. If so, the process 120 proceeds to the tangible design phase, as represented by block 138. Furthermore, it should be noted that the steps illustrated in FIG. 6 do not necessarily have to be performed in the order shown. Moreover, certain steps may be combined or skipped entirely depending on the aspects of the present disclosure.

[0044] 7 is a process flow diagram illustrating a process 150 for implementing the physical design step 32 of process 12 according to the present disclosure. In the illustrated embodiment of the process 150 beginning at start block 151, a centralized parallel approach is depicted. In the first branch of the process 150, as represented by block 152, a detailed audio, video, lighting, and projection (AVLP) design is prepared. This step may include preparation of artistic media, effects, and displays incorporating audio, video, lighting, and projection techniques. In the second branch, as represented by block 154, mechanical design may begin. This step may include mechanical simulation of mechanics (e.g., 3D modeling), determining the geometry of the shell or foundation that establishes the main structure (e.g., head skeleton housing) of the animated figure, determining the texture and / or structure of the skin attachment attached to (e.g., covering) the shell, etc. based on previously acquired data points such as updated computer generated models. Additionally, as represented by block 156, the basis established in block 154 may be used to determine physical positions for the joints and structures (e.g., the digital skeleton and joints of the updated computer-generated model), such as based on the steps performed in blocks 96, 98 of process 90. The actions performed in block 156 may also include simulated placement of real-world actuators in the updated computer-generated model (e.g., in the shell) to move the joints and / or structures based on the range of motion of the corresponding actuators, and determining associated displacements of aspects of the updated computer-generated model (e.g., structure, shell and / or skin movement) based on the action of the real-world actuators to simulate the movement of the animated figure. The simulated actuators in the computer-generated model may operate based on the animation data created and modified above, for example, to provide movement of the joints and structures of the computer-generated model.

[0045] As with other aspects of the disclosed techniques, these steps may be subject to iteration, such as to update selected mechanical functions based on joints and structures achievable via real actuators, as represented by block 158, and / or to update the projection simulation based on placement of real actuators. For example, the mechanical digital assets established by the acts represented by blocks 154, 156 may be utilized in process 90 of FIG. 5 (e.g., to indicate placement of components such as shells, real actuators, skin incorporated into the animated figure) to update the computer-generated assets completed in block 108. The iterations may continue until mechanical design approval is approved, as represented by block 160.

[0046] Once the mechanical design is approved (block 160), the process 150 proceeds to generate actuator and rigging related algorithms (e.g., models of joint and actuator location and action design), as represented by block 162, and then to detailed mechanical design, as represented by block 164. In particular, the generated algorithms and detailed mechanical design may include planning the assembly, positioning, and / or operation of the animated figure's shell, skin, structure, actuators, and features. As such, dimensions, sizes, tolerances, and other specifications associated with the animated figure may be determined based on the mechanical digital assets to facilitate preparation of the animated figure. Preparation of the animated figure may also include determining, positioning, arranging, and / or demonstrating indicia in and / or on the animated figure. Indicia may include signal emitters, uniquely colored materials (e.g., paints, dyes, and stickers), selected features, surface textures, pins, retroreflectors, lighting devices (e.g., visible or invisible infrared or ultraviolet light sources), and the like. Such indicia may be detectable using a camera or other sensor and may be utilized for synchronization of the animation figure and the projector so that the projection is properly aligned with the surface of the animation figure acting as the projection surface. Completion of the detailed mechanical design and completion of the AVLP design represents completion of the tangible design phase, as represented by block 166. At this point, process 150 proceeds to tangible fabrication, as represented by block 168.

[0047] FIG. 8 is a process flow diagram illustrating a process 180 for implementing the tangible fabrication / introduction step 34 of process 12 according to the present disclosure. As previously mentioned, a procedure such as process 180 includes steps illustrated in a particular order. However, it should be understood that the order of actions may be modified, reordered, truncated, etc., while remaining within the scope of the present disclosure. In the illustrated embodiment of process 180, a centralized parallel approach is depicted. The first branch of process 180 begins with the mechanical fabrication of the animated figure, as represented by block 182, which is performed using dimensions and specifications previously obtained from the mechanical digital assets. For example, block 182 may represent the fabrication of a shell (e.g., a head) for the animated figure. Block 184 represents the fabrication of the shell, skin, and any other suitable components (e.g., mechanisms or actuators) that may be implemented in the shell. Block 186 represents the assembly of the fabricated skin, shell, and mechanisms (e.g., actuators) together to form part of the tangible animated figure. Once the shell, skin, mechanisms and base are assembled (block 186), the actuation may be tested, tuned and adjusted, as represented by block 188, to achieve the desired aesthetics and functionality of the assembled parts of the animation figure based on the simulated actuation via the computer generated model. After the desired aesthetics and functionality of the assembled parts of the animation figure have been achieved, the completed mechanical model of the animation figure (e.g., a fully mechanically actuated head that may include a face configured to provide a variety of facial expressions) may be considered complete, as represented by block 190.

[0048] The second branch, which may proceed in parallel with the first branch, may begin with scene-specific production and preparation. Scene-specific production and preparation may include assembling the staging for the animated figures, associated props, and the like. Another step in this second branch includes the production and introduction of projections, cameras, and lighting, such as positioning cameras, projectors, lighting systems, and the like, to determine the appropriate look and feel for the scene, as represented by block 204. Once such features are positioned, a testing and adjustment step may occur, as represented by block 206. In particular, block 206 may include adjusting the positioning of cameras for use in identifying indicia positioned on the animated figures, as well as adjusting and synchronizing projectors and projections, and synchronizing the projections with the movements of the animated figures. Once the desired level of adjustment and tuning is reached, the scene-specific production may be considered complete, as represented by block 208. Once both branches of the process 180 are completed, the block process proceeds to character performance, as represented by block 210 in FIG. 8.

[0049] FIG. 9 is a process flow diagram illustrating a process 220 for performing the character performance step 36 of process 12 according to the present disclosure. Process 220, according to the present disclosure, involves the behavior of the animated figure (e.g., facial and / or figure animation) in conjunction with the projection onto the animated figure. Additionally, image data (e.g., image data used to project an image onto a tangible model, image data adjusted based on the image projected onto the tangible model) may be sent to a projector to project an image onto the animated figure. Such behavior of the projector may be adjusted to match the movement of the animated figure. Sound associated with the character performance may also be used during process 220, and the sound may be synchronized with relevant aspects of the character performance. Among other things, external tracking of the animated figure (e.g., via optical performance capture or optical motion capture) may be used by the present techniques and embodiments to dynamically generate and project images onto the exterior of the animated figure, thereby reducing or eliminating any unnatural background lighting. More specifically, to enhance the realism of the animated figure, the animated figure may be equipped with a tracker that enables a tracking camera of the media control system to recognize the motion, position, and orientation of the animated figure in real time via optical performance capture or optical motion capture. Thus, because the media control system may operate independently of the animated figure (e.g., by not relying on position, velocity, and / or acceleration information for the animated figure's actuators), the media control system may dynamically generate, adjust, and / or project images onto the interactive animated figure at realistic frame rates that mimic a live character, such as by reducing action latency and providing textures, colors, and / or motions that appear indistinguishable from the animated figure.

[0050] As illustrated in FIG. 9, step 220 may include applying animation data (e.g., programmed movements, movements corresponding to motion capture data, movements simulated via a computer-generated model), such as the animation data created in block 58 along with audio (when relevant) and iteratively modified based on the action of other processes, to the animation functions (e.g., facial expressions, or other figure movements) of the animated figure, as represented by block 222. The animation data may cause movement of the physical animated figure based on simulated actuation of the skeletal structure of the computer-generated model of the animated figure. As described above, the animation data may cause movement of the computer-generated model by activating simulated actuators of the computer-generated model. In addition, the animated figure may include real actuators that correspond to the simulated actuators. The animation data may thus be used to activate the real actuators to cause the corresponding movement of the animated figure. In this manner, the computer-generated model may be controlled to provide corresponding control of the animated figure. Also, specific animation data may be applied to specific actions, such as additional animation data that may supplement the animation data updated / modified in block 222, as broadly represented by block 224. For example, when facial actuation is used in conjunction with an animated figure, applying animation data via a direct algorithm to facial actuators may be used to adjust the movement of the animated figure.

[0051] Further, as represented by block 226, once the animation of the animated figure is finalized, the animation of the animated figure (e.g., the hands of the animated character) can be played back and synchronized with the projection (e.g., image data sent to the projector), tracking (e.g., a camera system tracking the marks of the animated figure), and AVLP to provide dynamic media. Testing and adjustment of the dynamic media (e.g., the movement of the animated figure, image data used to project the image) then occurs, as represented by block 228, and iteration of all or selected aspects of the process 220 may continue until evaluation of the performance of the animated figure and / or the projected image is finalized, as represented by block 230. For example, a simulated projector may be controlled to project a simulated image onto a computer-generated model of the animated figure, such that the appearance of the computer-generated model with the simulated image projected thereon closely corresponds to the appearance of the animated figure with the corresponding image projected thereon. Thereby, a real or physical projector may be controlled to project a corresponding image onto the animated figure based on the simulated image projected onto the computer-generated model. 9 and other figures depicting iterative procedures, in the illustrated embodiment, while block 230 and other iteration indicators may refer to specific steps, steps other than those shown, or even excluding the one shown, may be specified for the iteration. Such illustrations are representative of both broad and specific iteration options.

[0052] Of course, upon completion of process 220, the media control system of certain embodiments may generate and update a skeletal model of the animated figure based on feedback from the tracking camera. The skeletal model generally represents the actuatable parts of the animated figure, such as actuatable joints, and is dynamically updated to represent the current position, orientation, and / or scale of the animated figure or parts thereof (e.g., the pose of the animated figure) (e.g., including x, y, and z translation coordinates, and / or x, y, and z rotation coordinates). The media control system further utilizes the skeletal model to generate images for projection that precisely match the current position, orientation, and / or scale of the animated figure. The dynamic media system thereby provides a motion control system with the animated figure responsive to the interactive data as well as a media control system responsive to the mechanical performance of the animated figure. These two control loops thus provide improved system performance based on optical motion capture of the animated figure to deliver engaging character renditions to guests regardless of the mechanical positioning of the animated figure. Process 220 then moves to integration, as indicated by block 232.

[0053] 10 is a process flow diagram illustrating a process 240 for performing integration step 38 of process 12 according to an embodiment of the present disclosure. As illustrated in FIG. 10, process 240 may begin by introducing tangible assets (e.g., animated figures, lighting effects, camera systems, projectors, and support materials) in a scene area for an attraction system. This introduction process is represented by block 242. Once this introduction is completed, the process includes steps (as represented by blocks 244, 246, 248, 250, and 252) that may be performed iteratively by one or more controllers (e.g., automated controllers with one or more processors and memories, such as programmable logic controllers, etc.) according to the present disclosure. Block 244 represents in situ tracking and calibration of attraction effects (e.g., figure animation, projections, lighting, audio, and show control). For example, the actions and / or positioning of various components, such as animation figures (e.g., actuators), projectors, light emitters, sound emitters, etc., may be adjusted based on the introduction of the tangible asset, such as to accommodate tolerances or offsets as a result of the introduction. Block 246 represents a playback of the attraction effect, which may incorporate adjustments from the calibration, to determine the resulting appearance of the attraction effect. For example, a determination may be made as to whether the appearance of the dynamic media is as desired from the guest's perspective.

[0054] Block 248 represents digital adjustments to images (e.g., image data) for display (e.g., computer-generated, animated facial projection lighting and shading) on ​​the animated figure in the scene area. For example, the image may be adjusted based on the appearance of the image projected on the animated figure. Block 250 represents adjustments to lighting (e.g., stage lighting) in the scene area, such as to adjust visibility of various effects (e.g., parts of the animated figure) from the guest's viewpoint. Lighting may include real lighting, such as physical lighting that illuminates the scene area. Lighting may also include virtual lighting, which may include lighting that affects the projected image (e.g., lighting created by operating a projector or a display that provides an image). Lighting may be iteratively adjusted to make the projected image appear more realistic in the scene area, such as to appear as a physical feature instead of digital content. Block 252 represents synchronization and / or triggering of ride and control systems in coordination with other attractions. Synchronization of the ride and control systems may include controlling and calibrating the positioning of the vehicle along with certain movements and projections associated with the animated figure. For example, it may be desired to position the vehicle so that passengers have a clear and unobstructed line of sight to the animated figure during a particular phase of the presentation, and / or the relative positioning of the passengers and animated figure causes passengers to focus their gaze on a particular portion of the animated figure (e.g., the entire face). Block 254 represents the beginning of testing, adjustments, and iterations based on approval to determine whether the operation and positioning of certain components should be adjusted to provide the desired effect. If further adjustments are required based on testing (e.g., for a particular repeated action), such iterations may be repeated until approval is confirmed, and the process ends at block 256.

[0055] The technical effects of the disclosed dynamic media process include the provision of a motion control system with animated figures using one or more game engines, projection, motion capture, marker tracking, user input, artificial intelligence, modeling, actuator positioning, structural positioning, and various iterations to deliver an engaging and realistic experience to guests of the amusement attraction. In particular, the dynamic media process may include layering of various techniques to integrate subtle interactions that enhance thematic authenticity and immersion. The performance of the techniques (e.g., the performance of the iterations) may provide computer-generated devices that correspond to real models. The computer-generated models can be controlled to more closely and more accurately simulate the control of the real models. For example, the simulation of the movement of the computer-generated models and the projection of images onto the computer-generated models can provide a realistic appearance that corresponds to the movement of the real models and / or corresponds to the projection of images onto the real models. Such effects can provide more desirable show effects to guests. As an example, by manipulating an articulating jaw or other movable part of a physical animated figure in conjunction with projected media precisely coordinated for such motion through iterative simulation, production and synchronization, the present embodiment provides realistic and compelling effects. Moreover, the dynamic media process includes real-time tracking of markers and coordination with projections onto the outer surface of the animated figure. Thus, the process of the present disclosure can provide realistic projection / motion interactions based on a process that transitions between simulation and physical action to provide powerful alignment between different media types to mimic realism.

[0056] While only certain features of the disclosure have been illustrated and described herein, numerous modifications and changes will occur to those skilled in the art. It is therefore to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the present disclosure. Any of the features shown or described with respect to the above figures may be combined in any suitable manner.

[0057] The technology disclosed and claimed herein refers to and applies to tangible objects and examples of practical nature that clearly advance the art, not abstract, intangible, or merely theoretical. Moreover, if any claim appended at the end of this specification contains one or more elements specified as "means for [performing] [function]..." or "steps for [performing] [function]...", such elements are to be construed under 35 U.S.C. 112(f). However, for any claim that contains elements specified in any other manner, it is not intended that such elements be construed under 35 U.S.C. 112(f).

Claims

1. A method for generating dynamic media for expression via projection onto an animation figure, comprising: defining, via a processing circuit, a computer-generated model of the animation figure; operating, via the processing circuit, a manufacturing system to generate a physical model based on the computer-generated model; generating, via the processing circuit, a modified computer-generated model based on the physical model; simulating, via the processing circuit, projection of an image onto the modified computer model; operating, via the processing circuit, a projector to project the image onto the physical model based on the simulated projection of the image onto the modified computer-generated model. A method as described above.

2. receiving, via the processing circuit, an electronic scan of a real object; defining, via the processing circuit, the computer-generated model of the animation figure based on the electronic scan of the real object. A method according to Claim 1, including the above.

3. determining, via the processing circuit, a desired motion of the animation figure; defining, via the processing circuit, a digital skeleton structure of the computer-generated model based on the desired motion. A method according to Claim 1, including the above, wherein the digital skeleton structure includes the manner of connection and combination of motions of the computer-generated model.

4. receiving, via the processing circuit, motion capture data of a real object representing the computer-generated model; determining, via the processing circuit, the positioning of a first part of the real object based on the motion capture data; adjusting, via the processing circuit, the positioning of a second part of the computer-generated model corresponding to the first part of the real object in a 3D coordinate system based on the motion capture data; defining, via the processing circuit, the digital skeleton structure based on the adjustment of the positioning of the second part of the computer-generated model. A method according to Claim 3, including the above.

5. Defining, via the processing circuit, a simulated actuator in the computer-generated model based on the digital skeletal structure; Virtually actuating the simulated actuator via the processing circuit to cause an animation of the digital skeletal structure; The method according to claim 3, comprising:

6. Determining, via the processing circuit, a line of sight related to a visible portion of the animation figure; Generating, via the processing circuit, image data based on the line of sight; Simulating, via the processing circuit, a projection of the image onto the modified computer-generated model based on the image data; The method according to claim 1, comprising:

7. Adjusting, via the processing circuit, the image data based on the projection of the image onto the modified computer-generated model; Operating, via the processing circuit, the projector to project the image with the image data adjusted based on the projection of the image onto the modified computer-generated model; The method according to claim 6, comprising:

8. Obtaining, via the processing circuit, an electronic scan of the physical model; Generating, via the processing circuit, the modified computer-generated model based on the electronic scan; The method according to claim 1, comprising:

9. A system for dynamic media generation in an animation figure, comprising: A control system including a processing circuit and a memory, the memory storing instructions executed by the processing circuit to cause the control system to perform actions, the actions including: Defining, via the processing circuit, a computer-generated model of the animation figure; Operating, via the processing circuit, a manufacturing system to generate a physical model based on the computer-generated model; Generating, via the processing circuit, a modified computer-generated model based on the physical model; Simulating, via the processing circuit, a projection of an image onto the modified computer-generated model; Operating a projector to project the image onto the physical model based on a simulated projection of the image onto the modified computer-generated model via the processing circuit. A system comprising a control system including the above. **Claim 10** The control system is configured to perform operations including: Receiving a real object electronic scan via the processing circuit; Defining a computer-generated model of the animated figure based on the electronic scan of the real object via the processing circuit. The system according to claim 9, including the above. **Claim 11** The control system: Determining a desired motion of the animated figure via the processing circuit; Defining a digital skeleton structure of the computer-generated model based on the desired motion via the processing circuit, where the digital skeleton structure includes the connection mode and motion coupling of the computer-generated model. The system according to claim 9, configured to perform operations including the above. **Claim 12** The control system: Receiving motion capture data of a real object representing the computer-generated model via the processing circuit; Determining the positioning of a first position of the real object based on the motion capture data via the processing circuit; Adjusting the positioning of a second position of the computer-generated model corresponding to the first position of the real object in a 3D coordinate system based on the motion capture data via the processing circuit; Defining the digital skeleton structure based on the adjustment of the positioning of the second position of the computer-generated model via the processing circuit. The system according to claim 11, configured to perform operations including the above. **Claim 13** The control system: Defining a simulated actuator in the computer-generated model based on the digital skeleton structure via the processing circuit; Virtually operating the simulated actuator via the processing circuit to cause an animation of the digital skeleton structure. The system according to claim 11, configured to perform operations including the above. **Claim 14** The control system determines a line of sight associated with a visible portion of the animation figure via the processing circuit; generates image data based on the line of sight via the processing circuit; simulates projection of the image onto the modified computer-generated model based on the image data via the processing circuit; The system according to claim 9, configured to perform operations including the above. **Claim 15** The control system adjusts the image data via the processing circuit to provide adjusted image data based on the simulation of the projection of the image onto the modified computer-generated model; operates the projector via the processing circuit to project the image based on the adjusted image data; The system according to claim 14, configured to perform operations including the above. **Claim 16** One or more tangible, non-transitory, computer-readable media that, when executed by at least one processor, operate a manufacturing system to manufacture a physical model of an animation figure; generate a computer-generated model of the animation figure based on an electronic scan of the physical model; cause a first movement of the computer-generated model; cause a second movement of the animation figure based on the first movement of the computer-generated model; include instructions that cause the at least one processor to perform operations including operating a projector to output an image onto the animation figure based on the second movement of the animation figure. **Claim 17** The instructions, when executed by the at least one processor, define a simulated actuator corresponding to a physical actuator of the animation figure in the computer-generated model; actuate the simulated actuator to cause the first movement of the computer-generated model; Causing the at least one processor to perform an action including operating the physical actuator based on the operation of the simulated actuator to cause a second movement of the animated figure, the one or more tangible, non-transitory, computer-readable media of claim 16.

18. When the instructions are executed by the at least one processor, operating a simulated projector to output a simulated image to the computer-generated model, and operating the projector to output the image to the animated figure based on the output of the simulated image to the computer-generated model, the one or more tangible, non-transitory, computer-readable media of claim 16.

19. When the instructions are executed by the at least one processor, receiving user input for adjusting characteristics of a first computer-generated model of the animated figure, adjusting the first computer-generated model based on the user input, operating the manufacturing system to manufacture the physical model based on the adjustment of the first computer-generated model, receiving the electronic scan of the physical model, and adjusting the first computer-generated model based on the electronic scan of the physical model to generate a computer-generated model of the animated figure, the one or more tangible, non-transitory, computer-readable media of claim 16.

20. When the instructions are executed by the at least one processor, causing the one or more processors to monitor the positioning of the animated figure during the second movement, and adjust the illumination of the animated figure based on the positioning, the one or more tangible, non-transitory, computer-readable media of claim 16.

21. An amusement park attraction system, comprising a projector that outputs an image, A control system, at least one processor, defining a first computer-generated model corresponding to an animated figure, simulating the projection of the image onto the first computer-generated model to determine first simulated projection data, sending instructions to generate a first physical model based on the first computer-generated model and the first simulated projection data, generating an electronic scan of the first physical model, updating the first computer-generated model based on the electronic scan of the first physical model to obtain a second computer-generated model, simulating the projection of the image onto the second computer-generated model to determine second simulated projection data, sending instructions to generate a second physical model based on the second computer-generated model and the simulated projection data, wherein the first physical model and the second physical model are different physical models, a memory configured to store instructions configured to cause the control system to perform operations including, a control system including, an amusement park attraction system including.

22. The amusement park attraction system according to claim 21, wherein the second computer-generated model includes a plurality of simulated actuators, and the second physical model is configured to receive a plurality of physical actuators corresponding to the plurality of simulated actuators.

23. The control system, receiving animation data, actuating a simulated actuator among the plurality of simulated actuators of the second computer-generated model based on the animation data, actuating a physical actuator among the plurality of physical actuators based on the animation data and based on the physical actuator corresponding to the simulated actuator, The amusement park attraction system according to claim 22, configured to perform operations including.

24. The control system, receiving motion capture data, Generating the animation data based on the motion capture data; The amusement park attraction system according to claim 23, which is configured to perform the above.

25. The amusement park attraction system includes a guest area and a scene area, the second tangible model is arranged in the scene area, and the control system Determining the relative positioning between the guest area and the scene area; Operating the projector to output the image to the second tangible model based on the relative positioning; It is configured to perform operations including the above. The amusement park attraction system according to claim 21.

26. The guest area includes a vehicle configured to move relative to the scene area, and the control system operates the vehicle to be configured to move relative to the scene area. The amusement park attraction system according to claim 25.

27. The amusement park attraction system includes lighting, and the control system Monitoring the positioning of the first tangible model; Adjusting the lighting based on the positioning of the first tangible model; The amusement park attraction system according to claim 21, which is configured to execute instructions including the above.

28. The first tangible model is a miniaturized model of the second tangible model. The amusement park attraction system according to claim 21.

Images