System and method for media projection of animated figures

Abstract

A method of generating dynamic media for representation by projection onto an animated figure includes defining a computer-generated model of the animated figure via a processing circuit, operating a manufacturing system via the processing circuit to generate a tangible model based on the computer-generated model, generating a modified computer-generated model via the processing circuit based on the tangible model, simulating the projection of an image onto the modified computer-generated model via the processing circuit, and operating a projector via the processing circuit to project an image onto the tangible model based on the projection of the simulated image onto the modified computer-generated model.

Description

TECHNICAL FIELD

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS)] This application claims priority and benefit to U.S. Application No. 63 / 212,375, filed June 18, 2021, entitled "Systems and Methods for Animated Figure Media Projection", the citation of which is incorporated into this application for all purposes.

BACKGROUND ART

[0002] This section is intended to introduce the reader to various aspects of technologies that may be related to various aspects of the present techniques described and / or claimed below. It is believed that this description will help provide the reader with background information to facilitate a better understanding of various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this right, and not as an admission of prior art.

[0003] Amusement parks and other entertainment venues often include animated figures (e.g., robot characters) among their many attractions to entertain guests while they wait in line for rides or during the ride experience. Certain animated figures can be brought to life through projection mapping, which conventionally directs a predetermined appearance onto the figure. For example, certain animated figures may be visually supplemented with a set of pre-recorded or fixed images that can be positioned to match the figure's pre-programmed movements. While such techniques can be more entertaining than flat display surfaces, their development is now recognized as being able to further immerse guests in specific attractions, rides, or interactive experiences. For example, certain animated figures may have internally positioned projectors that generate surreal background lighting or emit light through internal or rear projections via the figure's translucent projection surface. Thus, it is now recognized that it is desirable not only to provide animated figures with the ability to blend them into the environment and context in a realistic and compelling manner, but also to make the animated figures appear more lifelike.

[0004] (summary) Specific embodiments equivalent to the scope of the initially claimed title are summarized below. These embodiments are not intended to limit the scope of the Disclosure, and are intended only to provide a brief summary of certain disclosed embodiments. Furthermore, the Disclosure encompasses a variety of forms that may be similar to or different from the embodiments described below.

[0005] In one embodiment, dynamic media generation for performance by projection onto an animated figure includes: defining a computer-generated model of an animated figure by a processing circuit; operating a manufacturing system by the processing circuit to generate a tangible model based on the computer-generated model; generating a modified computer-generated model by the processing circuit based on the tangible model; simulating the projection of an image onto a received computer-generated model by the processing circuit; and operating a projector by the processing circuit to project an image onto a tangible model based on the projection of an image onto the simulated modified computer-generated model.

[0006] In one embodiment, an amusement park attraction system includes a projector configured to output an image, and a control system configured to operate the projector to output an image to the second tangible model, which defines a first computer-generated model corresponding to an animated figure, simulates the projection of an image onto the first computer-generated model to determine a first simulated projection data, sends commands to generate a first tangible model based on the first computer-generated model and the first simulated projection data, updates the first computer-generated model to obtain a second computer-generated model based on an electronic scan of the first tangible model, simulates the projection of an image onto a second computer-generated model to determine a second simulated projection data, sends commands to generate a second tangible model based on the second computer-generated model and the second simulated projection data.

[0007] In one embodiment, one or more tangible, non-temporary, computer-readable media include instructions that, when executed by at least one processor, cause at least one processor to operate a manufacturing system to manufacture a tangible model of an animated figure; generate a computer-generated model of the animated figure based on an electronic scan of the tangible model; cause a first motion of the computer-generated model; cause a second motion of the animated figure based on the first motion of the computer-generated model; and operate a projector that outputs an image to the animated figure based on the second motion of the animated figure.

[0008] These and other features, modes and advantages of this disclosure will be better understood when modes for carrying out the following invention are read with reference to drawings of similar features attached thereto that represent similar parts thereto. [Brief explanation of the drawing]

[0009] [Figure 1] This is an overview diagram of a system that provides dynamic media in the manner of this disclosure. [Figure 2] This is a block diagram of a system providing dynamic media and the corresponding system operation as described herein. [Figure 3] This is a process flow diagram showing the process for implementing the character asset steps of the process operation shown in Figure 2, in the manner of this disclosure. [Figure 4] This is a process flow diagram showing the process for performing the process analysis steps of the process action shown in Figure 2, in the manner of this disclosure. [Figure 5] This is a process flow diagram representing the process animation / rigging steps of the process operation shown in Figure 2, in the manner of this disclosure. [Figure 6] This is a process flow diagram showing the process for performing the verification step of the process operation shown in Figure 2, in the manner of this disclosure. [Figure 7]This is a process flow diagram showing the process for implementing the tangible design steps of the process operation shown in Figure 2, in the manner of this disclosure. [Figure 8] This is a process flow diagram showing the process for carrying out the tangible manufacturing / introduction steps of the process operation shown in Figure 2, in the manner of this disclosure. [Figure 9] This is a process flow diagram representing the process that executes the character performance steps of the process operation shown in Figure 2, in the manner of this disclosure. [Figure 10] This is a process flow diagram representing the process that executes the integrated steps of the process operation shown in Figure 2, in the manner of this disclosure. [Modes for carrying out the invention]

[0010] One or more specific embodiments of this disclosure are described below. In an effort to provide a concise description of these embodiments, not all features of actual implementations are described herein. It should be recognized that in the development of any such actual implementation, as in any optical or design project, a great many implementation-specific decisions must be made to achieve the developer's specific goals, such as adhering to constraints of system and business relationships that may differ from one implementation to another. Moreover, while such development efforts tend to be complex and time-consuming, they can be considered routine work for those skilled in the art in designing, fabricating, and manufacturing.

[0011] When introducing elements of various embodiments of the present invention, the articles "a," "an," and "the" indicate the presence of one or more elements. The terms "equip," "include," and "have" are comprehensive and indicate that additional elements other than those described may exist. Furthermore, it should be understood that references to "one embodiment," "one embodiment," or "several embodiments" of the present invention are not intended to be interpreted as excluding the existence of additional embodiments incorporating similarly described features.

[0012] Theme parks and other such entertainment venues continue to grow in popularity. Furthermore, there is a strong demand for immersive experiences within these entertainment venues. Thus, there is a recognized need to enhance the thematic effect to provide a more immersive experience for guests, and to make visitors truly magically engrossed in the themed environment. One of the more challenging aspects of providing such immersion involves creating animated figures (frequently used for themed entertainment) that look more lifelike than traditional methods can achieve. Unrealistic animated figures can cause a disruption of immersion. In particular, providing the faces of characters (e.g., humans) to animated figures is perceived as especially difficult for visitors to adapt to the highly detailed aspects of facial structure and interaction, which can be perceived as subtly unrealistic. Problems identified alongside traditional methods, and that prevent the deliberate cessation of disbelief, include the insufficient illusion of life provided only by three-dimensional (3D) forms (e.g., robotics), the operation of artificial skin that only represents a part of a realistic expression, and projections that can be easily noticed based on the emission of light from within the animated figure. Therefore, the embodiments are directed toward control modes that coordinate the pre-processing of projection media, external projection onto tangible mechanical assets (e.g., animated figures of robots), and the layering of effects that will provide a realistic experience. As described below, certain modes and procedures are carried out based on centralized or decentralized processor-based control, along with steps associated with physical transitions and actions based on automation and / or human interaction.

[0013] From the standpoint of the foregoing, this embodiment is generally directed toward a method and system for creating media projections onto animated figures for amusement attractions. In particular, this embodiment is directed toward providing attraction features that facilitate immersion in a theme by providing realistic, lifelike, or other immersive media projections onto the outer surface of the animated figure in conjunction with other controlled modes of the animated figure. Layered interactions (e.g., projected animation in conjunction with the operation of a fabricated 3D projection surface and the animated figure) provide projection media, control features, sound, and modes of the animated figure. Such layered interactions or actions provide effects that can be noticed by the observer, in a manner that is highly realistic and subtly differentiated. Such layering is partially achieved by utilizing several interaction data sources or data streams (e.g., the current state of the animated figure, the quality of the guest's actions or responses, the actions of performers backstage or on stage) to generate detailed, complex, and subtle effects that correspond to reality. Due to the complex nature of these interactions, in some embodiments, control commands from artificial intelligence (AI) or machine learning engines can adjust media, projection position, animation speed, projection surface position, etc., to create a realistic and immersive experience. Furthermore, to progressively improve the production process, a large number of variable and repeatable routines are included, repeated until a recognized level of realism or immersion is achieved.

[0014] In particular, the techniques disclosed herein provide apparatus and foundations for exposing what may be referred to herein as dynamic or active media. Dynamic media is defined herein as media that layers the provision of a dynamic and immersive experience to guests in which animated figures resemble real people or characters more closely than conventionally realized. In particular, providing dynamic media as described herein includes projection mapping images onto the outer surface of an animated figure, in conjunction with preparing / operating the components of the animated figure and based on various inputs, thereby providing an animated figure that appears more lifelike. Projection mapping can be particularly beneficial when compared to certain animated figure systems that project images into the interior through the translucent surfaces of the animated figure, resulting in unnatural background lighting or a glistening effect. Furthermore, dynamic media such as projected images and / or motion of an animated figure can be interactive or responsive, such as based on interaction with guests or other perceived parameters. Thus, instead of providing predetermined show effects, show effects may be dynamically generated and thereby vary with respect to differences in the cycle of action, etc., in order to provide guests with a more immersive experience.

[0015] As described herein, embodiments and techniques define the foundations that enable the preparation of dynamic media, using character definition, performance analysis, animation rigging, projection control, animation figure control, animation figure movement, etc. As described herein, these aspects are integrated or layered to provide a total effect of optical, physical, or audible interactions based on a large amount of input and control (e.g., show control, controlling various multimedia controllers). This total effect is the foundation for providing dynamic media and relates to an immersive experience for the viewer. For example, the results of the techniques described herein may provide a computer-generated model that more precisely and accurately corresponds to a physical, tangible, real animation figure, and the computer-generated model may be controlled (e.g., to give movement to the computer-generated model) to facilitate the corresponding control of the physical animation figure and to facilitate more sensitive control over the physical animation figure and the image show effects projected thereon.

[0016] While the examples provided herein generally focus on mimicking the head or face of a character (e.g., a human character), it should be understood that embodiments are not declared in such examples. Furthermore, any method of an animated character may be represented by these embodiments. For example, features not typically considered to be faces may be utilized and referred to herein as faces (e.g., the front of a car or a factory may have features that enhance or add to the impression of a human face). Some modes or embodiments may still be clearly perceived as mimicking actual facial features, and such of human characters may be perceived as difficult in the relevant area due to the subject's familiarity with and adaptation to such features. Moreover, as used herein, animated figures may include any suitable physical objects or props that can move within the space occupied by the objects. Furthermore, animated figures may include clothing (e.g., shirts worn by the user), deformable objects (e.g., balloons that may inflate and / or deflate), balls that may be rolled and / or thrown, or any other suitable objects that can move, and the images projected onto the objects may be dynamically adjusted based on such motion of the objects. Although this disclosure is primarily described in the context of amusement parks, the techniques and embodiments described herein can be applied to medical, transportation, architectural, or any other applications in which computer-generated models and corresponding physical models or figures may be used.

[0017] The procedures provided by this disclosure (applicable to the procedures illustrated in Figure 2-10) for dynamic media include a variety of different steps and modes of procedure. Some of these steps or procedures may be performed in parallel or in different order. Some steps may be processor-based actions and may involve controlled equipment (e.g., actuators or 3D printers). Furthermore, some procedures may be performed repeatedly to achieve the desired outcome. Thus, while the steps of various different procedures may be described in a particular order herein, the steps of a procedure may not have to be performed in the order presented herein. While some specific steps of an action may always occur before other specific steps (e.g., as commanded by logic), the enumeration of certain orders of actions is provided primarily for ease of explanation. For example, indicating that the first step or initiation step includes a particular action is not intended to limit the scope of this disclosure to such an initial step. Rather, it should be understood that additional steps may be performed, certain steps may be omitted, and referenced steps may be performed in an appropriate alternative order or in parallel. However, the order in which the effects are disclosed may be restricted when presented in this manner.

[0018] Figure 1 shows a system 10 implementable for creating and providing dynamic or active media according to this disclosure. System 10 may include a control system 11 which may include control circuits (e.g., one or more automation controllers, one or more electronic controllers, one or more programmable controllers) configured to implement various techniques described herein for providing dynamic media. Control system 11 may include one or more processors or processing circuits 14 and tangible computer-readable media 18. Tangible computer-readable media 18 may include volatile memory such as random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM), optical devices, hard disk devices, solid-state drives, or any other suitable non-temporary computer-readable media in which instructions can be stored. Processor 14 may be configured to execute instructions. For example, processor 14 may include one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more general-purpose processors, or any combination thereof.

[0019] The control system 11 can be configured to operate dynamic media to provide a desired experience in order to give guests a show effect. For example, system 10 may include a scene or stage area 1 in which a projector 2, an animated figure 3, and other props 4 can be positioned. The control system 11 can be configured to operate projector 2 to output content or images to the animated figure 3 in order to provide a desired (e.g., realistic) appearance of the animated figure 3. The control system 11 can also be configured to generate the movement of the animated figure 3, for example, through the action of actuators 5, in order to control the movement of the animated figure 3. Furthermore, the control system 11 may adjust the action between projector 2 and the animated figure 3 to adjust the image output by the movement of projector 2 and the animated figure 3 relative to each other, in order to give a realistic appearance or effect that may not be easily achieved otherwise. The control system 11 can also be configured to operate other props 4, such as light emitters or sound emitters, fog / smoke effect systems, etc., in order to complete or supplement the effects provided through the action of projector 2 and / or the animated figure 3. In one embodiment, the control system 11 may be communicably connected to a sensor 21 configured to determine various operating parameters, and the control system 11 may be configured to operate based on these operating parameters. For example, the operating parameters may include the position and orientation of the animated figure 3 and / or the projector 2 in the scene area, sound output in the scene area (e.g., by other props 4), and / or the appearance of the image projected onto the animated figure 3.

[0020] A guest may be positioned in the guest area 6 of the system 10, and may be able to view the effects provided by the scene area 1. As an example, the scene area 1 and the guest area 6, which may operate to provide a unique experience to guests, may be part of an attraction system of an amusement park. The guest area 6 may include a vehicle that can move relative to the scene area 1 (e.g., along a track or path), and guest passengers of the vehicle can view the scene area 1 while the vehicle passes through the scene area 1. Additionally or alternatively, such as for a dramatic performance, the guest area 6 may be fixed relative to the scene area 1 and may include an auditorium-style arrangement. In any case, the operation of the scene area 1 via the control system 11 may provide dynamic media that entertains the guests in the guest area 6.

[0021] The control system 11 can also be configured to repeatedly perform various operations to generate desired dynamic media, such as facilitating the creation, production, operation, deployment, etc. of the projector 2, the animated figure 3, and / or the prop 4. For example, the control system 11 can be configured to acquire desired character assets to generate a model representing the animated figure 3. In certain embodiments, the control system 11 can be configured to operate a scanning system 7, such as a 3D scanning device, to capture an image of a real object 8 and create a computer-generated model 9 based on the image. The control system 11 can receive the computer-generated model 9 from the scanning system 7. The computer-generated model 9 may be used to assist in the design and creation of the animated figure 3. For example, the control system 11 may include a display 13 utilized by a user to view the computer-generated model 9, and the control system 11 may include a user interface (UI) 15, such as a mouse, keyboard, touch screen, trackpad, dial, buttons, etc., that the user can use to adjust various features of the computer-generated model 9.

[0022] The control system 11 can also be configured to operate a manufacturing system 17, such as a 3D printing machine and / or a modeling machine (e.g., an injection molding machine), to create a tangible model 19. The tangible model 19 may be a more simplified version of the animation figure 3 (e.g., a part of the animation figure, a downsized model of the animation figure) for the purpose of performing initial evaluation and / or testing. As an example, before starting a more complex design and / or manufacturing process for the animation figure 3, an initial determination may be made as to whether the image output to the tangible model 19 (e.g., via the projector 2) looks desirable. In some embodiments, the computer-generated model 9 may be adjusted based on the appearance of the tangible model 19 (e.g., the image projected onto the tangible model 19). For example, the control system 11 can operate the scanning system 7 to create the computer-generated model 9 based on the tangible model 19 (e.g., an image captured of the tangible model 19). Thus, the computer-generated model 9 can be updated to represent the tangible model 19 more precisely in place of the real object 8. Such a computer-generated model 9 can also be adjusted via the control system 11, and subsequently used to create the next tangible model 19 via the manufacturing system 17.

[0023] In this method, a great many iterations of the computer-generated model 9, and / or the tangible model 19, can be created until a desired appearance sample of the animated figure 3, along with the image projected thereon, is produced. When a satisfactory appearance (for example, of the tangible model) is achieved, the control system 11 can facilitate the production of the animated figure 3. For example, the control system 11 can complete the computer-generated model 9 by creating a design more suitable for a mechanical manufacturing process. The final product of the animated figure 3 may then be produced based on the completed computer-generated model 9 and prepared for introduction into the scene area 1. The actions of the animated figure 3, such as motion supported by the actuator 5, may be based on the computer model 9 (for example, motion simulated via the computer-generated model 9).

[0024] Figure 2 shows a system 10 capable of performing a process 12 for providing dynamic media as described herein. Process 12 can be an example of various computer operations performed via a processor 14 based on instructions 16 stored in tangible computer-readable media, including operations using AI and machine learning. Some operations may not be directly controlled by the processor (e.g., certain physical operations). However, even certain physical operations, such as manufacturing (e.g., by a 3D printer), involve a mode of control. Each step of process 12 is described in more detail below.

[0025] The start of Process 12 is represented by a start block 20 that indicates the start of Character Assets Step 24, which may provide characters or elements on which the projection media may be based, and Performance Analysis Step 26, which may provide scenes, venues, narratives, environments, settings, and / or functions on which the animated figure may perform. Character Assets Step 24 and Performance Analysis may be performed in parallel or sequentially. The results from these steps are then used in Animation / Rigging Step 28, which can generate a computer-generated model representing the animated figure on which the media will be projected. The following Animation / Rigging Step 28 and Verification Step 30 are performed to complete the computer-generated model of the animated figure, and Tangible Design Step 32 may then be performed to prepare the animated figure for production, such as providing mechanical digital assets based on the completed computer-generated model. Animation / Rigging Step 28, Verification Step 30, and Tangible Design Step 32 include iterative modes to refine the results to a desired level of audience-perceived reality. Once the tangible design operation 32 is designated as perfect, the tangible production / implementation step 34 is carried out to create a stage or environment on which the animated figure may be implemented to provide the animated figure and / or dynamic media. The character performance step 36, which determines the movement of the animated figure and / or the projection of images onto the animated figure, is then carried out to coordinate the movement of the animated figure and the action of the dynamic media followed by the integration step 38 in order to introduce the dynamic media into the scene area and complete the action of the dynamic media. Once the integration step 38 is completed, the entire process 12 may be considered to have ended with the completed dynamic media generated and created as the product of process 12, as the end block 40 is designated. The resulting dynamic media may then be put into operation in an attraction system of an amusement park system and given to guests.For example, the completion of integration step 30 may provide a computer-generated model and a physical animated figure that closely corresponds to the computer-generated model. The action of dynamic media may include using the computer-generated model to simulate the motion of the computer-generated model and / or the image projected onto the computer-generated model, and providing the physical animated figure and / or the corresponding motion of the image projected onto the physical animated figure. Details related to the individual aspects of the overall process 12 are described in more detail below.

[0026] Figure 3 is a process flow diagram representing process 50 carrying out the character asset step 24 of process 12 as disclosed herein. Character asset step 24, sometimes referred to as character asset creation, includes the development or selection of a character asset, which becomes the preparation of a character or projection media that displays a projection of the character or the appearance of the character as a component of providing dynamic media (e.g., projection media for representing an animation of a human face). Process 50 is started as indicated by start block 52 and proceeds first to character determination as represented by block 54. Character determination may include selecting a specific subject (e.g., a person, a sculpted image, an animal). In selecting a character, a model or form of the character (e.g., a 3D digital human face model) may be created as represented by block 56. The form or model (e.g., a 3D model) may be referred to as a computer-generated model. For example, a predefined or predetermined model (e.g., a 3D model) or computer-generated model may exist for the selected subject, and the predefined model may be selected and retrieved from a storage device. In addition, or instead, new computer-generated models may be defined for characters, such as for subjects that do not have associated models and / or for subjects that may have older models. For example, 3D scanning techniques use 3D scanning technology (typically with very high-precision data acquisition) to create 3D computer-generated models of real-world objects such as statues, objects, or living creatures. In particular, 3D scanners capture a large number of images of an object and generate an electronic scan (e.g., an electronic mapping of the subject). The captured images are merged, stitched together, joined together, or otherwise integrated into a 3D computer-generated model that is an electronic 3D replica of the subject or object, which may include various points or parts of the subject mapped onto a 3D coordinate system.Once a computer-generated model is defined (e.g., via scanning), a software application can, in relation to it, allow viewing and / or editing the computer-generated model on a display (e.g., a computer screen) from various different angles. For example, a software application can facilitate virtually rotating the computer-generated model on the display (e.g., using a user interface such as a mouse or keyboard).

[0027] In some embodiments, the computer-generated model can be graphically modified, as shown in Block 57. For example, the color, texture, and features of the computer-generated model (e.g., facial features, hair, ornaments, or accessories) can be added, removed, or otherwise adjusted. The model ratio or size of the computer-generated model may also be modified. In this method, the computer-generated model can be easily customized to achieve a desired depiction, such as based on the implementation of the projected media.

[0028] As shown in Figure 3, process 50 proceeds to define the character performance as represented in block 58. Defining the character performance may be performed before, during, or in parallel with the creation of the computer-generated model (block 56) and / or modification of the computer-generated model (block 57). The steps shown in block 58 may include capturing real-world video and other motion data of the actor's character performance, including relevant facial feature movements (e.g., movements used by the subject while acting out a scene). Motion capture technology may be used to capture such video images and motion data. The data associated with this captured video can then be used together with the computer-generated model to define the character performance. Specifically, motion capture technology and data point concatenation may be used to define the character performance. For example, motion capture points on the actor's lips may be identified for later use in the display of the lips in the computer-generated model (e.g., an electronic face model) along with corresponding points (e.g., coordinate points), and the relevant motion data from the motion capture may be applied to the lips in the computer-generated model. In other words, the motion of a computer-generated model (for example, in a 3D coordinate system) may be determined based on motion capture of the corresponding subject represented by the computer-generated model. As another example, 3D animation techniques may be used to define character performance. In particular, software applications may be used to edit and manipulate underlying images and / or computer-generated models to create a series of images that are adjusted to provide animation. Thus, further manipulation of the computer-generated model can define information representing the group of characters. In this embodiment, a great many conventional animation techniques may be used.A character's defined animation or movement (e.g., the character's position over time) may be stored as animation data that can be readily retrieved for implementation and / or additional modifications. Furthermore, audio may be recorded as part of or separately from the visual portion of the character's performance. The audio can be combined with the video data to provide a complete character performance. However, in some situations, video data may be generated and / or used without audio data.

[0029] Using the acquired data, process 50 may generate or define digital assets (e.g., fixed face models) that may be stored, acquired, and / or further manipulated to perform different motions and / or provide different appearances, as represented by block 60. Data relating to data, arrays of position points, or configurations (e.g., facial expressions) may be used to define digital assets (e.g., digital face assets) that can represent the appearance of a character at a specific point in time. This step may involve using a structural construction algorithm (e.g., a face construction algorithm) that utilizes available data to predict and adapt the manner of a certain type of structure (e.g., facial structure) to facilitate the depiction of a wide range of motion-based actions (e.g., facial expressions). Furthermore, this procedure enables the generation and simulation of various instances of the representation without requiring the specific incorporation of such motions in the preceding step. In addition, the appearance development step may be performed on this digital asset to prepare the relevant graphics for use with a game engine, which may include software tools or application programming interfaces that facilitate the construction of a graphics-based model of a subject or character, as represented by block 62. Specifically, using conventional techniques, this may include texturing, face modeling, and assembly of geometry instances to facilitate animation that is realistic or otherwise immersive for use in providing dynamic media. In this regard, the illustrated process 50 may be considered the end. However, as will be discussed below, digital face assets (represented, for example, by block 28 in Figure 1 and by block 64 in Figure 3) may be used later, along with animation and / or rigging, to provide a finished computer graphics asset (for example, after the completion of appearance development).

[0030] Figure 4 is a process flow diagram representing process 70, which performs the performance analysis step 26 of process 12 according to this disclosure. Data obtained from process 70 may be used to facilitate the hierarchical arrangement of effects that provide dynamic media. For example, according to embodiments of this disclosure, timing, position, and orientation data obtained during process 70 may be used in a concert to control media displays, model actions, and the actions of characters that have been given movement (e.g., the actions of a robot). The start of process 70 is represented by a start block 72, and as previously stated, process 70 may be performed before, after, or in parallel with the character asset step 24.

[0031] As the first step in process 70 in Figure 4, the scene or venue-specific staging of the character is determined to determine the implementation of the character in the scene area, which may include other show effects, props, or elements in addition to the character (step 74). Scene-fixed staging may be implemented using computer modeling (including computer-generated models of the scene area and animated figures), which may include mapping specific coordinates of the character in different poses or orientations (including modes of volumetric analysis) within a 3D coordinate system of the scene area at different points in time. Furthermore, as represented by block 76, line of sight and field of view can be determined or calculated for the view of one or more guests. Determining line of sight and field of view may include determining the mean or average view for a great many different audience viewpoints, such as based on the positional adjustment of the guest and / or guest area relative to the real scene area (e.g., via a computer-generated model). For example, the movement or positioning of the character in the scene area may be determined based on line of sight and field of view to bring the visibility of the target part of the character to the guest. Furthermore, as represented by Block 78, the duration of the scene (e.g., the total allocated time, or the total time generally used to perform a particular scene or expression) is determined. Determining the duration of a scene may include determining the timeframes of the scene's aspects or parts, such as the total duration of running a single cycle of the scene, or timestamps indicating when the scene begins and / or ends. Determining the duration of a scene may also include, in addition to or instead of, determinations related to the characters in the scene, such as the individual durations related to maintaining various positional adjustments of the characters and thereby maintaining the visibility resulting from different parts of the characters, timeframes for performing movements by the characters (e.g., transitions between facial expressions), or timestamps indicating the characters' movements relative to the movements of other elements of the scene (e.g., when a character moves in conjunction with the movement of additional props).

[0032] The process may also include an initial projection and tracking / camera design, which may include a virtual or physical placement of related devices, such as a projector and / or device (e.g., a bogie) that may move the projector, as represented by block 80. The initial projection and / or tracking design may simulate the appearance (e.g., from the guest's perspective) of the image output to the character through the projector, from specific positional adjustments of the projector (e.g., position, four directions). Furthermore, the process may include formal learning based on cone / frustum projection, angles, field of view, guest's line of sight, etc., as represented by block 82. Projection learning may be performed via software to help determine whether the image projected onto the character is distorted, obstructed (e.g., blocked or obscured by other props), or otherwise undesirable output by the projector. Thus, projection learning may be used to determine whether the image data used to output the image should be adjusted, whether the projector positional adjustments should be changed, whether a different projector or type of projector should be used, whether additional projectors should be incorporated, etc.

[0033] In block 84, adjustments to projection adjustments based on character motion and configuration transitions may be provided by sensing certain character configuration states (e.g., geometry, contours). For example, for a character with a dynamic or active mouth (e.g., a character physically capable of transitioning mouth features between open and closed configurations), open-mouth configurations (e.g., a configuration where the character's jaw is open) and / or closed-mouth configurations (e.g., a configuration where the character's jaw is closed) may be determined. In other embodiments, different configurations (e.g., an upturned or downturned forehead configuration) may be determined. Furthermore, larger or more significant transitions (e.g., morphing of a ball, a star shape, or a spiky ball) may also be applied. Again, such data may be used to facilitate the stratification of effects that create realistic images by associating different projected images, types of projected images (e.g., images projected by a certain projector or from a certain direction), or categories of projected images (e.g., a happy representation of an open-mouth configuration, as a happy representation of a closed-mouth configuration is compared) with different possible configurations of the character. For example, the configuration of a physical character's open mouth can be associated with a projection that aligns with the configuration of an enlarged character's head (relative to the geometry of a closed mouth), creating a more realistic and immersive viewing experience. Furthermore, it is believed that the results provided by this embodiment should be partially superior to conventional representations due to the layering of such effects and the subtle details associated with such layering. Similar to process 50 shown in Figure 3, the data generated in process 70 may now be used, along with animation and / or rigging (as shown in Figure 1 by block 28 and in Figure 4 by block 86), to provide a finished computer graphics asset.

[0034] Figure 45 is a process flow diagram representing process 90 performing the animation / rigging step 28 of process 12 according to this disclosure. The animation / rigging step according to embodiments of this disclosure includes skeletal animation techniques (e.g., by a computer modeling program) for representing a 3D character model (e.g., a computer-generated model created in block 54 of process 50) using a series of virtually connected digital structural features, which may be referred to as a digital skeleton. In particular, animation / rigging (or simply “rigging”) may be used in this disclosure to refer to a process step that creates a digital skeletal structure of a 3D character model in order to define the manner in which different parts of the 3D character model are connected and the combination of motion. This digital skeletal structure may be used according to this embodiment to manipulate a doll-like 3D character model for animation purposes. Such animation may include manipulating and distorting the face model to produce different expressions.

[0035] Process 90 may begin with block 92, which may include preliminary rigging from character assets assembled in process 50 (e.g., digital face assets created in block 60). In other words, the initial digital skeletal structure of the character assets may be created to define the initial set of linked modes of the character assets. However, in some situations, the initial digital skeletal structure may be more complex, obscure, or excessive than desired for the specific motion and / or projection mapping purposes related to the implementation of the character. As shown in Figure 5, the next step, represented by block 94, may involve finalizing animation goal changes (e.g., rough animation) which may include applying character performance (e.g., character performance defined in block 58 of process 50, such as video-captured character performance) to appearance development (e.g., appearance development completed in block 62 of process 50). In this technique, the animation data created and stored as a result of block 58 may be modified (e.g., iteratively modified) based on appearance development and other steps performed to enable the animation data to produce simulated motion of the digital skeletal structure. Process 90 may further include a structural analysis (e.g., facial analysis) based on the positional adjustment of structural features (e.g., eyebrows, temples, cheeks, mouth, chin, lips) in a main orientation or positional adjustment (e.g., facial expression), as represented by block 96. Using data from this analysis, the main orientation (e.g., main facial expression) of the digital skeleton and digital joints (e.g., the linkage between digital skeletons) may be determined. Thus, a specific subset of digital skeletons and digital joints may be determined to operate or move separately to form different main orientations via animation data, for example, to change a 3D character model between different representations of a corresponding fixed face model for a given scene.

[0036] Next, the 3D character model (e.g., a computer-generated model of a human face) is simplified (e.g., using a simplification algorithm) to create a more amorphous 3D character model in terms of form while manipulating the main structure, as shown in Block 98. Thus, certain digital skeletons and / or digital joints that are not used to achieve different main orientations to a scene may be removed to reduce the amount of the set of connected modes of the character asset. For example, a face model may be simplified (e.g., certain connected facial structures, such as between the ear and jawbone, may be removed or made unrelated to each other) and made more amorphous based on an algorithm designed to produce such simplifications limited to certain expressions. Simplification of 3D character models can be performed manually (e.g., the user can manually select whether digital skeletons and / or digital joints should be removed or retained) and / or automatically (e.g., based on automatic sensing that the usage of certain digital skeletons and / or digital joints falls below a threshold in order to provide the main orientation). The fact that the position and connections between the digital skeleton and digital joints remain unchanged facilitates the identification and maintenance of the main facial structures in such algorithms in order to achieve the primary orientation.

[0037] Once a simplified 3D character model is generated in block 98, process 90 can proceed to an iterative portion of process 90. This may include creating computer graphics devices and mechanical function lists for specific actions (e.g., facial functions), as represented by block 100. That is, a list of different resulting outputs may be determined, such as the movement of the 3D character model (e.g., the simplified character model) associated with activating or moving different maintained digital skeletons and / or digital joints of the 3D character model (e.g., the simplified character model) to achieve the movement of the 3D character model associated with animation data created in block 58, and / or the appearance of the 3D character model (e.g., facial expressions). Thus, the initially generated mechanical function list may contain many different types of outputs, such as more orientations than the main orientation desired for implementation. In addition, the iterative portion may include selecting, adjusting, modifying, or identifying the mechanical function list, as represented by block 102. For example, specific actions of the digital skeletons and / or digital joints of the 3D character model can be determined to be used to achieve the main orientation via animation data. Thus, the initially generated mechanical function list may be simplified to reduce the amount of mechanical functions included in the mechanical function list in order to focus on what is desirable to provide the main direction. In one embodiment, animation data related to the motion of a 3D character model may be updated based on the mechanical function list, for example, to provide updated motion that may be more compatible or achievable via a digital skeletal structure (e.g., digital skeleton, digital joints) in terms of the mechanical function list.

[0038] Furthermore, as represented by block 104, the iterative process may include projection simulations of computer-generated images (via a processor-based computer simulator, or other software, such as computer graphics compositing and visual effects software) that may take into account previously determined data, including character appearance development, guest gaze, movement, and / or the main orientation of the 3D character model. The projection simulation may generate a sample of image data of virtual content or images that, when projected onto the 3D character model, provides a realistic depiction of the projected appearance onto a physical, real-world object sample of the 3D character model. For example, for projection simulations performed by a processor-controlled simulator (e.g., a simulation package or application running on a computer), and / or computer graphics compositing and visual effects software, the 3D character model (e.g., a 3D character model of a face) may be positioned (e.g., rotated) relative to a virtual projector, and the quality of the simulated lens relative to the virtual projector may be virtually selected to match those actual projectors. The simulation can then draw the projection surface (e.g., structure and texture) relative to the virtual projector and output the resulting image. When the projection surface and virtual projector adequately correspond to real-world relative objects, the result can more accurately represent the appearance of the content projected in reality.

[0039] The subsequent iterations of these procedures depend on the output from determination block 106, which includes a determination of whether the computer-generated image output from process 90 is acceptable in this respect (for example, based on user input or algorithm-based computer implementation criteria). If the output is unacceptable (for example, the main orientation caused by certain digital skeletal movements and / or digital joints of the 3D character model is undesirable, the appearance of the image projected onto the 3D character model is undesirable, or the adjustment between the movement of the 3D character model and the adjusted image projected from the virtual projector is undesirable), the iteration may be restarted in any one of blocks 100, 102, and / or 104. The selection of the appropriate procedural step for the subsequent iterations may depend on data related to the computer-generated image output at the relevant time. For example, a decision may be made regarding whether the movements of the digital skeleton and / or joints of the 3D character model should be adjusted based on a mechanical function list, whether the image output by the virtual projector should be adjusted (for example, by adjusting the image data to output an image to the virtual projector), or both based on the quality of the 3D character model together with its appearance and / or projected image. The evaluation of 3D character models and projected images may be based on algorithms, artificial intelligence, and / or user input. Once the output computer-generated image is deemed sufficient in block 106, a completed computer-generated asset is defined, which may include a computer-generated 3D character model, as represented by block 108. In this regard, block 110 brings about a change to the verification process, as shown in Figure 5.

[0040] Figure 6 is a process flow diagram representing process 120, which carries out the verification process 30 of process 12 according to this disclosure. Verification process 120 may begin with the production of a tangible or physical version of the computer-generated model (e.g., 3D printing of the character's head and face), as shown in block 122. This may include automated 3D printing, AM technology, modeling (e.g., injection molding), or component production (e.g., cutting) based on commands from a controller, after verification of a well-made computer-generated image. The production of the tangible model may include improvement actions to facilitate the use of the manufactured model for projection testing. For example, improvement actions may include sanding, polishing, texturing, painting, trimming, punching, cutting, etc. Next, projection testing may be performed using multiple fixed poses of animation in the manufactured model, as shown in block 124. For example, projection testing may include projecting images of different facial expressions onto a manufactured facial structure model. For this purpose, as in block 104 of process 90, image data used to simulate image projection onto a computer-generated model and / or image data adjusted based on the projection onto the simulated computer-generated model may be sent to a projector to output a tangible model image based on the image data. In one embodiment, various parts of the tangible model (e.g., a puppet rig) may be movable relative to one another to enable at least partial achievement of the main orientation to the character represented by the tangible model. However, it should be noted that the tangible model created during verification step 30 may be a more simplified design (e.g., having a limited range of motion) with respect to the completed tangible animated figure for provision and implementation in dynamic media.

[0041] Next, various iterative actions may be performed, as shown by the iteration and approval block 126. As shown in Figure 6, various steps may be included as iterative actions. In particular, in some steps, physically or manually operable features may be incorporated, as shown by block 128. For example, block 128 may represent the addition of flexible skin to the fabricated head, along with actuators (e.g., hand-operable levers) to provide facial movement. In other embodiments, other features of the animated figure may be updated, corresponding to flexible skin (e.g., fur, scales, or fur) and associated actuators. Another iterative step may include the incorporation of servo motors and controllers for automatic synchronized motion of associated actions, as shown by block 130. For example, actuators associated with facial features may be operated automatically (e.g., without a user manually operating each individual actuator) to display a series of expressions on the fabricated head and face, along with the flexible skin, in various synchronized movements. It should be noted that the steps in blocks 128 and 130 may be mixed, removed, or kept separately in accordance with the manner of this disclosure.

[0042] Another iterative step may include block 132, representing a projection onto the tangible model assembled in blocks 128 and / or 120, in order to determine the correspondence between the mechanical and projected characteristics of the tangible model (e.g., alignment of eye movements with mechanical positioning). In this regard, an iterative step, such as that represented by block 134, may include scanning the tangible model (e.g., using a LiDAR (light detection and ranging) or 3D scanner) to determine an updated computer-generated model (e.g., an electronic scan or mapping) based on the tangible model. The preceding steps, such as generating a skeletal structure, determining a list of mechanical functions, and / or simulating the projected image, may be performed using such an updated or new computer-generated model. For example, the motion, positioning, and / or geometry of the tangible model may be acquired, such as through motion capture of the tangible model, and such motion, positioning, and / or geometry may be used to update animation data used to control the computer-generated model. In this method, certain steps of the aforementioned process, such as process 90 performed earlier for the initially created computer-generated model, may be performed for the computer-generated model updated based on the tangible model. Such action, which initially provides or utilizes a simpler tangible model, improves the efficiency of iteration without requiring multiple designs and the creation of more complex and complete animated figures, and facilitates the transformation from the initially created computer-generated model (e.g., via block 56 of process 50) to the working physical structure.

[0043] Furthermore, as illustrated in Figure 6, a decision can be made whether to iterate further to provide an updated computer-generated model, or to complete the computer-generated model, by iterating and updating and producing the next tangible model and / or adjusting the image data projected onto the tangible model, using data from the previous step in the iteration. In particular, a decision can be made as to whether a living function of the projection onto the form of the tangible model is recognized, as represented by block 136. If recognized, process 120 proceeds to the tangible design phase, as represented by block 138. It should also be noted that the steps described in Figure 6 do not necessarily have to be performed in the order shown. Furthermore, certain steps may be combined or skipped entirely, depending on the form of this disclosure.

[0044] Figure 7 is a process flow diagram representing process 150 carrying out the tangible design step 32 of process 12 according to this disclosure. In the illustrated embodiment of process 150, which begins in introductory block 151, a concentrated parallel approach is depicted. In the first branch of process 150, as represented by block 152, detailed sound, video, lighting, and projection (AVLP) design is prepared. This step may include preparing artistic media, effects, and displays that incorporate sound, video, lighting, and projection techniques. In the second branch, as represented by block 154, mechanical design may begin. This step may include mechanical simulation of mechanics (e.g., 3D modeling) based on previously acquired data points such as an updated computer-generated model, determining the geometry of a shell or base that establishes the main structure of an animated figure (e.g., a head skeleton housing), determining the texture and / or structure of skin accessories attached to (e.g., covering) the shell. Furthermore, as represented by block 156, the physical position of joints and structures (e.g., the digital skeleton and joints of the updated computer-generated model) can be determined using the basis determined in block 154, based on the steps performed in blocks 96 and 98 of process 90. The actions performed in block 156 may also include the simulated placement of real-world actuators in the updated computer-generated model (e.g., within a shell) for moving joints and / or structures based on the range of motion of the corresponding actuators, and the determination of the relevant displacements of the updated computer-generated model (e.g., movement of structures, shells and / or skin) based on the actions of real-world actuators for simulating the motion of an animated figure. The simulated actuators in the computer-generated model may operate based on the animation data created and modified above, for example, to provide the motion of the joints and structures of the computer-generated model.

[0045] As with other aspects of the techniques disclosed, these steps can be the subject of iteration, such as updating mechanical functions selected based on joints and structures achievable via real actuators, and / or updating projection simulations based on the arrangement of real actuators, as represented by block 158. For example, to update the computer-generated assets completed in block 108, the mechanical digital assets determined by the actions represented by blocks 154 and 156 may be used in process 90 of Figure 5 (e.g., to show the arrangement of components such as the shell, real actuators, and skin incorporated into the animated figure). Iteration may continue until mechanical design approval is granted, as represented by block 160.

[0046] Once the mechanical design is approved (block 160), process 150 proceeds to the generation of actuator and rigging-related algorithms (e.g., models for joint and actuator position and action design), and then to the detailed mechanical design, as shown in block 162, and then in block 164. In particular, the generated algorithms and detailed mechanical design may include assembling, positioning, and / or planning the operation of the animation figure's shell, skin, structure, actuators, and features. Therefore, dimensions, size, tolerances, and other specifications related to the animation figure may be determined based on the mechanical digital assets to facilitate the preparation of the animation figure. The preparation of the animation figure may also include determining, positioning, arranging, and / or marking the animation figure's interior and / or surface. Markings may include signal emitters, uniquely colored materials (e.g., paints, dyes, and stickers), selected features, surface textures, pins, retroreflectors, and lighting devices (e.g., visible or invisible infrared or ultraviolet light sources). Such markings may be detectable using a camera or other sensor and may be used to synchronize the animation figure and projector so that the projection is properly positioned on the surface of the animation figure that will act as the projection surface. The completion of the detailed mechanical design and the AVLP design constitutes the completion of the tangible design phase, as represented by block 166. At this point, process 150 proceeds to tangible fabrication, as represented by block 168.

[0047] Figure 8 is a process flow diagram representing process 180, which carries out the tangible fabrication / introduction step 34 of process 12 as described herein. As stated above, a procedure like process 180 includes steps illustrated in a specific order. However, it should be understood that the order of operations may be modified, rearranged, truncated, etc., while remaining within the scope of this disclosure. In the illustrated embodiment of process 180, a centralized parallel approach is depicted. The first branch of process 180 begins with the mechanical fabrication of the animated figure, as represented by block 182, which is carried out using dimensions and specifications obtained in advance from mechanical digital assets. For example, block 182 may represent the fabrication of the shell (e.g., head) for the animated figure. Block 184 represents the fabrication of the shell, skin, and any other suitable components (e.g., mechanisms or actuators) that may be implemented in the shell. Block 186 represents the assembly of the fabricated skin, shell, and mechanisms (e.g., actuators) into one another to form a part of the tangible animated figure. Once the shell, skin, mechanism, and base are assembled (block 186), the operation may be tested, synchronized, and adjusted, as shown in block 188, based on simulated operation via a computer-generated model, in order to achieve the desired aesthetic and functional aspects of the assembled parts of the animated figure. After the desired aesthetic and functional aspects of the assembled parts of the animated figure are achieved, the completed mechanical model of the animated figure (e.g., a fully mechanically operated head that may include faces configured to produce various expressions) may be considered complete, as shown in block 190.

[0048] The second branch, which may proceed in parallel with the first branch, may begin with scene-specific production and preparation. Scene-specific production and preparation may include assembling the staging for animated figures, associated props, etc. Another step in this second branch is the production and introduction of projection, camera, and lighting, such as positioning cameras, projectors, lighting systems, etc., to determine the appropriate look and feel for the scene, as represented by block 204. Once such features are positioned, testing and adjustment steps may occur, as represented by block 206. In particular, block 206 may include adjusting the camera position for the purpose of identifying markings positioned on the animated figures, and synchronizing the projection with the movement of the animated figures, along with adjusting and synchronizing the projector and projection. Once the desired level of adjustment and tuning is reached, scene-specific production may be considered complete, as represented by block 208. Once both branches of process 180 are complete, the block process proceeds to character performance, as represented by block 210 in Figure 8.

[0049] Figure 9 is a process flow diagram representing process 220, which performs the character performance step 36 of process 12 according to this disclosure. Process 220, according to this disclosure, involves the action of the animated figure (e.g., facial and / or figure animation) in conjunction with projection onto the animated figure. Furthermore, image data (e.g., image data used to project an image onto a tangible model, image data adjusted based on the image projected onto the tangible model) may be sent to a projector to project an image onto the animated figure. Such action of the projector may be adjusted to match the movement of the animated figure. Audio related to the character performance may also be used during process 220, and the audio may be synchronized with the manner in which the character performance is involved. In particular, external tracking of the animated figure (e.g., via optical performance capture or optical motion capture) is used by this technique and embodiments to dynamically generate images and project them onto the outer surface of the animated figure, thereby reducing or eliminating any unnatural background lighting. More specifically, to enhance the realism of the animated figure, the animated figure may be equipped with trackers that allow the media control system's tracking camera to recognize the animated figure's movement, position, and orientation in real time via optical performance capture or optical motion capture. Thus, the media control system can operate independently of the animated figure (for example, by not relying on position, velocity, and / or acceleration information relating to the animated figure's actuators), and can dynamically generate, adjust, and / or project images onto the interactive animated figure at a realistic frame rate that mimics a living character, by reducing latency and providing texture, color, and / or motion that appears indistinguishable from the animated figure.

[0050] As illustrated in Figure 9, step 220 may include applying animation data (e.g., programmed motion, motion corresponding to motion capture data, motion simulated via a computer-generated model) to the animation functions of the animated figure (e.g., facial expressions, or motion of other figures), such as animation data created in block 58 (with sound, where relevant) and iteratively modified based on the action of other processes, as represented by block 222. The animation data may cause physical motion of the animated figure based on the simulated operation of the skeletal structure of the computer-generated model of the animated figure. As described above, the animation data may cause motion of the computer-generated model by operating the simulated actuators of the computer-generated model. In addition, the animated figure may include real actuators that correspond to the simulated actuators. The animation data may therefore be used to operate the real actuators and cause corresponding motion of the animated figure. In this method, the computer-generated model may be controlled to provide corresponding control of the animated figure. Furthermore, as broadly represented by block 224, certain animation data, such as additional animation data that can complement the updated / modified animation data in block 222, may be applied to specific actions. For example, when facial expressions are used with an animated figure, the step of applying animation data to the facial actuator via a direct algorithm may be used to adjust the movement of the animated figure.

[0051] Furthermore, as shown by block 226, once the animation of the animated figure is finalized, the animation of the animated figure (e.g., the hands of an animated character) can be played back to provide dynamic media, and synchronized with projection (e.g., image data sent to a projector), tracking (e.g., a camera system that tracks the markers of the animated figure), and AVLP (Audio Visual Live Program) operations. As shown by block 228, testing and adjustment of the dynamic media (e.g., the movement of the animated figure, image data used to project the image) then occurs, and as shown by block 230, iterations of all or selected modes of process 220 may continue until the performance of the animated figure is evaluated and / or the projected image is finalized. For example, a simulated projector may be controlled to project a simulated image onto a computer-generated model of the animated figure, and the appearance of the computer-generated model with the simulated image projected onto it closely corresponds to the appearance of the animated figure with the corresponding image projected onto it. Thereafter, a real or physical projector may be controlled to project a corresponding image onto the animated figure based on the simulated image projected onto the computer-generated model. With respect to Figure 9 and other diagrams representing iterative procedures, in the illustrated embodiments, block 230 and other iteration indicators may refer to specific steps, while steps other than those shown, or further excluding one of those shown, may be designated for iteration. Such illustrations are representative of both broad and specific iteration options.

[0052] Naturally, in relation to the completion of process 220, a media control system of a certain embodiment may generate and update a skeletal model of the animated figure based on feedback from a tracking camera. The skeletal model generally represents the movable parts of the animated figure, such as movable joints, and is dynamically updated to represent the current position, orientation, and / or scale of the animated figure or a part thereof (e.g., the pose of the animated figure), including x, y, and z translation coordinates and / or x, y, and z rotation coordinates. The media control system further utilizes the skeletal model to generate an image for projection that precisely matches the current position, orientation, and / or size of the animated figure. The dynamic media system thereby provides a motion control system having an animated figure that responds to interactive data, as well as a media control system that responds to the mechanical performance of the animated figure. These two control loops, as a result, provide improved system performance based on optical motion capture of the animated figure to deliver engaging character performances to guests regardless of the mechanical positioning of the animated figure. Next, process 220 moves on to integration, as shown by block 232.

[0053] Figure 10 is a process flow diagram representing process 240 performing integration step 38 of process 12 according to embodiments of the present disclosure. As illustrated in Figure 10, process 240 may be initiated by introducing tangible assets (e.g., animated figures, lighting effects, camera systems, projectors, and support materials) in a scene area for an attraction system. This introduction process is represented by block 242. Once this introduction is complete, the process may be repeated by one or more controllers (e.g., automated controllers having one or more processors and memory, such as programmable logic controllers) according to the present disclosure, including steps (represented by blocks 244, 246, 248, 250, and 252). Block 244 represents tracking in place and calibration of attraction effects (e.g., figure animation, projection, lighting, sound, and show control). For example, the operation and / or positional adjustment of various components, such as animated figures (e.g., actuators), projectors, light emitters, and sound emitters, may be adjusted based on the introduction of tangible assets, such as to adapt tolerances or offsets as a result of the introduction. Block 246 represents the reproduction of an attraction effect, which may incorporate adjustments from calibration to determine the appearance as a result of the attraction effect. For example, a decision may be made regarding whether the appearance of dynamic media is as desired from the guest's field of view.

[0054] Block 248 represents digital adjustments to an image (e.g., image data) for display on an animated figure within a scene area (e.g., lighting and shading of computer-generated, animated face projections). For example, the image may be adjusted based on the appearance of the image projected onto the animated figure. Block 250 represents adjustments to lighting (e.g., stage lighting) within a scene area, such as adjusting the visibility of various effects (e.g., parts of an animated figure) from the guest's perspective. Lighting may include actual lighting, such as physical lighting illuminating the scene area. Lighting may also include virtual lighting, such as lighting that affects the projected image (e.g., lighting created by operating a projector, or a display providing an image). Lighting may be iteratively adjusted in the scene area so that the projected image looks more realistic, such as appearing as a physical feature instead of digital content. Block 252 represents synchronization and / or triggering of ride and control systems in conjunction with other attractions. Synchronization of the ride and control systems may include controlling and calibrating the ride's positioning, along with specific movements and projections related to the animated figures. For example, during certain phases of the performance, it may be desirable to position the ride so that the passenger has a clear, unobstructed view of the animated figures, and / or so that the relative positioning of the passenger and the animated figures causes the passenger to focus their gaze on a specific part of the animated figure (e.g., the entire face). Block 254 represents the start of iterations based on testing, adjustments, and approval to determine whether the action and positioning of certain components should be adjusted to provide the desired effect. If further adjustments are required based on testing (e.g., with respect to a particular iterative action), such iterations may be repeated until approval is confirmed, and the process ends in Block 256.

[0055] The technical effects of the disclosed dynamic media process include supplying a motion control system with one or more game engines, projection, motion capture, marker tracking, user input, artificial intelligence, modeling, actuator positioning, structural positioning, and animated figures using various iterations, in order to deliver an engaging and realistic experience to guests of an amusement attraction. In particular, the dynamic media process may include the layering of various techniques that integrate subtle interactions to enhance the credibility and immersion of the theme. The performance of the techniques (e.g., the performance of iterations) may provide computer-generated devices that correspond to real-world models. Computer-generated models can be controlled to more rigorously and accurately simulate the control of real-world models. For example, the simulation of the motion of a computer-generated model, and the projection of images onto the computer-generated model, can provide a realistic appearance that corresponds to the motion of a real-world model and / or the projection of images onto a real-world model. Such actions can provide guests with a more desirable show effect. As an example, by manipulating a jointed jaw, or other movable parts of a physical animated figure, in conjunction with projection media precisely adjusted to such motion through iterative simulation, production, and synchronization, this embodiment provides a realistic and compelling effect. Furthermore, the dynamic media process includes real-time tracking of markers and coordination with projection onto the outer surface of the animated figure. Thus, the process of this disclosure can provide realistic projection / motion interaction based on a process that transitions between simulation and physical action, providing strong positional adjustments between different types of media to mimic reality.

[0056] While only certain features of the disclosure are illustrated and described herein, those skilled in the art will be able to conceive of numerous modifications and variations. Therefore, it should be understood that the appended claims are intended to cover all such modifications and variations so as to fall within the essential scope of this disclosure. Any features illustrated or described above for the figures may be combined in any suitable manner.

[0057] The technologies disclosed and claimed herein refer to and apply to tangible objects and specific examples of practical characteristics that clearly enhance the art, such as those that are not abstract, intangible, or merely theoretical. Furthermore, if any claims attached to the end of this specification include one or more elements designated as “means for carrying out the [function]…” or “steps for carrying out the [function]…”, such elements should be construed under § 112(f) of the United States Patent Act. However, for any claims that include elements designated in any other manner, such elements are not intended to be construed under § 112(f) of the United States Patent Act.

Claims

1. A method for generating dynamic media for expression via projection onto an animated figure, The process involves defining a computer-generated model of the animated figure via a processing circuit, The manufacturing system is operated via the processing circuit to generate a tangible model based on the computer-generated model. The process circuit generates a modified computer-generated model based on an electronic scan of the tangible model, The processing circuit simulates the projection of an image onto the modified computer model and provides a depiction of the image on the modified computer-generated model. The processing circuit is used to operate the projector to project the image onto the tangible model based on the simulated projection of the image onto the modified computer-generated model. Methods that include...

2. The processing circuit receives an electronic scan of a real object, The processing circuit defines the computer-generated model of the animated figure based on the electronic scan of the real object. The method according to claim 1, including the method described in claim 1.

3. The desired motion of the animated figure is determined via the processing circuit, Through the processing circuit, the digital skeletal structure of the computer-generated model is defined based on the desired motion. The method according to claim 1, wherein the digital skeletal structure includes the coupling of the computer-generated model and the coupling of its motion.

4. The processing circuit receives motion capture data of a real object representing the computer-generated model, The processing circuit determines the positioning of the first part of the real object based on the motion capture data, The processing circuit adjusts the positioning of the second part of the computer-generated model, which corresponds to the first part of the real object in the 3D coordinate system, based on the motion capture data. The digital skeletal structure is defined based on the adjustment of the positioning of the second part of the computer-generated model via the processing circuit, The method according to claim 3, including the method described in claim 3.

5. Through the processing circuit, the simulated actuator is defined in the computer-generated model based on the digital skeletal structure, The processing circuit is used to virtually operate the simulated actuators in order to produce animation of the digital skeletal structure. The method according to claim 3, including the method described in claim 3.

6. The processing circuit determines the line of sight related to the visible portion of the animated figure, The processing circuit generates image data based on the gaze direction, The processing circuit simulates the projection of the image onto the modified computer-generated model based on the image data. The method according to claim 1, including the method described in claim 1.

7. The image data is adjusted via the processing circuit based on the projection of the image onto the modified computer-generated model. The process circuit is used to operate the projector to project the image using the image data adjusted based on the projection of the image onto the modified computer-generated model. The method according to claim 6, including the method described in claim 6.

8. The electronic scan of the tangible model is acquired via the processing circuit, The process circuit generates the modified computer-generated model based on the electronic scan. The method according to claim 1, including the method described in claim 1.

9. A system for generating dynamic media in animated figures, A control system including a processing circuit and a memory, wherein the memory stores instructions to be performed by the processing circuit in order to cause the control system to perform an action, and the action is, The processing circuit defines a computer-generated model of the animated figure, The manufacturing system is operated via the processing circuit to generate a tangible model based on the computer-generated model. The process circuit generates a modified computer-generated model based on an electronic scan of the tangible model, The processing circuit simulates the projection of an image onto the modified computer-generated model and provides a depiction of the image on the modified computer-generated model. The process circuit is used to operate a projector to project the image onto the tangible model based on the simulated projection of the image onto the modified computer-generated model. A system including a control system.

10. The control system is configured to perform an action, and the action is The processing circuit receives an electronic scan of a real object, The processing circuit defines the computer-generated model of the animated figure based on the electronic scan of the real object, The system according to claim 9, including the system described in claim 9.

11. The control system is The desired motion of the animated figure is determined via the processing circuit, The process circuit defines a digital skeletal structure of the computer-generated model based on the desired motion, wherein the digital skeletal structure includes the coupling of the computer-generated model and the coupling of motion. The system according to claim 9, configured to perform an action including the action described above.

12. The control system is The processing circuit receives motion capture data of a real object representing the computer-generated model, The processing circuit determines the positioning of the first part of the real object based on the motion capture data, The processing circuit adjusts the positioning of the second part of the computer-generated model corresponding to the first part of the real object in the 3D coordinate system based on the motion capture data, The digital skeletal structure is defined via the processing circuit based on the adjustment of the position of the second part of the computer-generated model, The system according to claim 11, configured to perform an action including the action described above.

13. The control system is Through the processing circuit, the simulated actuator in the computer-generated model is defined based on the digital skeletal structure, The processing circuit is used to virtually operate the simulated actuator and trigger animation of the digital skeletal structure. The system according to claim 11, configured to perform an action including the action described above.

14. The control system is The processing circuit determines the line of sight associated with the visible portion of the animated figure, The processing circuit generates image data based on the gaze direction, The processing circuit simulates the projection of the image onto the modified computer-generated model based on the image data, The system according to claim 9, configured to perform an action including the action described above.

15. The control system is The processing circuit adjusts the image data to provide adjusted image data based on the simulation of the projection of the image onto the modified computer-generated model. The processing circuit is used to operate the projector and project the image based on the adjusted image data, The system according to claim 14, configured to perform an action including the action described above.

16. One or more tangible, non-transient, computer-readable media, when executed by at least one processor, To manufacture tangible models of animated figures, the manufacturing system is operated, Based on the electronic scan of the tangible model, a computer-generated model of the animated figure is generated. To generate the first motion of the computer-generated model, Based on the first motion of the computer-generated model, a second motion of the animated figure is generated, One or more tangible, non-transient, computer-readable media, including an instruction to cause at least one processor to perform an action including operating a projector to output an image to the animated figure based on the second movement of the animated figure.

17. When the instruction is executed by the at least one processor, In the computer-generated model, define simulated actuators corresponding to the physical actuators of the animated figure, To operate the simulated actuator to produce the first motion of the computer-generated model, One or more tangible, non-transient, computer-readable media according to claim 16, wherein the at least one processor is caused to perform an action including acting on the physical actuator based on the action of the simulated actuator in order to produce a second motion of the animated figure.

18. When the instruction is executed by the at least one processor, The simulated projector is operated to output the simulated image to the computer-generated model, One or more tangible, non-transient, computer-readable media according to claim 16, wherein the at least one processor is made to perform an action including operating the projector to output the image to the animated figure based on the output of the simulated image to the computer-generated model.

19. When the instruction is executed by the at least one processor, Receiving user input to adjust the features of the initial computer-generated model of the aforementioned animated figure, Adjusting the initial computer-generated model based on the user input, Based on the adjustment of the initial computer-generated model, the manufacturing system is operated to manufacture the tangible model. Receiving the electronic scan of the tangible model, One or more tangible, non-transient, computer-readable media according to claim 16, wherein the at least one processor is made to perform an operation including adjusting the initial computer-generated model based on the electronic scan of the tangible model in order to generate the computer-generated model of the animated figure.

20. When the instruction is executed by the at least one processor, the one or more processors will: During the second motion, the positioning of the animated figure is monitored, One or more tangible, non-transient, computer-readable media according to claim 16, which perform an action including adjusting the lighting of the animated figure based on the positioning.

21. It is an amusement park attraction system. A projector that outputs images, A control system, At least one processor, To define the first computer-generated model corresponding to the animated figure, To simulate the projection of the image onto the first computer-generated model and to determine the first simulated projection data, Sending instructions to generate a first tangible model based on the first computer-generated model and the first simulated projection data, To generate an electronic scan of the first tangible model, Based on the electronic scan of the first tangible model, the first computer-generated model is updated to obtain a second computer-generated model. To simulate the projection of the image onto the second computer-generated model and to determine the second simulated projection data, Sending commands to generate a second tangible model based on the second computer-generated model and the simulated projection data, wherein the first tangible model and the second tangible model are different tangible models from each other. A memory configured to store instructions configured to cause the control system to perform an action including the above, A control system including, An amusement park attraction system, including [this].

22. The amusement park attraction system according to claim 21, wherein the second computer-generated model includes a plurality of simulated actuators, and the second tangible model is configured to receive a plurality of physical actuators corresponding to the plurality of simulated actuators.

23. The control system is Receiving animation data and, Based on the animation data, the simulated actuator among the plurality of simulated actuators in the second computer-generated model is activated, Based on the animation data and based on the physical actuators corresponding to the simulated actuators, the physical actuators among the plurality of physical actuators are activated. The amusement park attraction system according to claim 22, configured to perform an action including the following.

24. The control system is Receiving motion capture data, Based on the motion capture data, the animation data is generated, The amusement park attraction system according to claim 23, configured to carry out the following.

25. The amusement park attraction system includes a guest area and a scene area, the second tangible model is placed in the scene area, and the control system is Determining the relative positioning between the guest area and the scene area, The projector is operated to output the image to the second tangible model based on the relative positioning, It is configured to perform an action that includes, The amusement park attraction system according to claim 21.

26. The amusement park attraction system according to claim 25, wherein the guest area includes a ride vehicle configured to move relative to the scene area, and the control system is configured to operate the ride vehicle to move relative to the scene area.

27. The amusement park attraction system includes lighting, and the control system is Monitoring the positioning of the first tangible model, Based on the positioning of the first tangible model, the lighting is adjusted, The amusement park attraction system according to claim 21, configured to carry out instructions including

28. The amusement park attraction system according to claim 21, wherein the first tangible model is a miniaturized model of the second tangible model.

Images