Generating interactive digital experiences using a real-time 3D rendering platform

Abstract

To provide a method, computer-readable medium and system for creating interactive digital experiences for linear content.SOLUTION: The method includes: identifying a plurality of assets relating to presentation of linear content; generating interactive content using the linear content, including generating an interactive sequence referencing one or more of the assets; and combining the linear content with the interactive sequence on a timeline sequentially describing the linear content. The timeline includes one or more branches relating to the linear content, and selection of a first branch of the one or more branches is based on the interactive sequence. The method further includes transmitting the interactive content to a user.SELECTED DRAWING: Figure 3

Description

Cross-reference to Related Applications

[0001] This application claims the benefit and priority of U.S. Patent Application No. 17 / 725,648, filed on April 21, 2022, which in turn claims the benefit and priority of U.S. Provisional Patent Application No. 63 / 178,803, filed on April 23, 2021. These related patent applications are hereby incorporated by reference in their entirety. Background

[0002] Adding interactive elements to linear content (e.g., a television program or a movie) is difficult. As will be described later, linear content is non-branching, does not provide interactivity or provides very limited interactivity, and provides a limited user experience. On the other hand, conventional television games include interactive elements but are typically executed using state machine or level loading techniques that may not preserve the production and creative intent of the content creator. There is a need for a solution that efficiently combines the interactivity of television games with non-branching linear content while preserving the production and creative intent of the linear content creator. Summary

[0003] Embodiments include a method. The method includes identifying a plurality of assets related to the presentation of linear content. Further, the method includes generating interactive content using the linear content, the generating including generating an interactive sequence that references one or more of the plurality of assets, and combining the linear content and the interactive sequence on a timeline that sequentially describes the linear content. The timeline includes one or more branches related to the linear content. Selection of a first branch of the one or more branches is based on the interactive sequence. Further, the method includes transmitting the interactive content to a user.

[0004] Furthermore, the embodiment includes a non-temporary computer-readable medium containing computer program code that performs the operation when executed by an operation of one or more computer processors. The operation includes the step of identifying a plurality of assets related to the presentation of linear content. Furthermore, the operation includes the step of generating interactive content using the linear content, which includes the step of generating an interactive sequence that references one or more of the plurality of assets, and the step of combining the linear content and the interactive sequence on a timeline that sequentially describes the linear content. The timeline includes one or more branches related to the linear content. The selection of a first branch of the one or more branches is based on the interactive sequence. Furthermore, the operation includes the step of sending the interactive content to a user.

[0005] Furthermore, the embodiment includes a computer processor and memory that stores instructions for performing an operation when executed on the computer processor. The operation includes the step of identifying a plurality of assets related to the presentation of linear content. Furthermore, the operation includes the step of generating interactive content using the linear content, which includes the step of generating an interactive sequence that references one or more of the plurality of assets, and the step of combining the linear content and the interactive sequence on a timeline that sequentially describes the linear content. The timeline includes one or more branches related to the linear content. The selection of a first branch of the one or more branches is based on the interactive sequence. Furthermore, the operation includes the step of sending the interactive content to a user. [Brief explanation of the drawing]

[0006] To enable a detailed understanding of the embodiments described herein, which have been briefly summarized above, a more detailed description of these embodiments can be provided by referring to the accompanying drawings.

[0007] However, it should be noted that the accompanying drawings illustrate a typical embodiment and are therefore not to be construed as limiting, and other equally valid embodiments are intended. [Figure 1] This figure shows a computing environment for generating interactive digital experiences for linear content, according to one embodiment. [Figure 2] This figure shows a controller for generating interactive digital experiences for linear content, according to one embodiment. [Figure 3] This flowchart shows the generation of an interactive digital experience for linear content according to one embodiment. [Figure 4] Block diagram showing an interactive digital experience generated for linear content according to one embodiment. [Figure 5] This figure further illustrates a world-space interaction for generating an interactive digital experience for linear content, according to one embodiment. [Figure 6] This is a flowchart of a timeline for generating an interactive digital experience for linear content according to one embodiment. [Figure 7A] This figure shows a reference between a linear and an interactive timeline for creating an interactive digital experience for linear content, according to one embodiment. [Figure 7B] This figure further illustrates a reference between linear and interactive timelines for creating interactive digital experiences for linear content, according to one embodiment. Detailed explanation

[0008] Aspects of this disclosure provide apparatus, methods, processing systems, and computer-readable media for creating interactive digital experiences for linear content. As described herein, linear content is non-branching digital content for presentation to a user (e.g., a movie or television program). In embodiments, predefined digital linear content assets can be combined with real-time generated content (e.g., content generated using a real-time game engine) to create an interactive experience that enables continuous user engagement. For example, linear content can be combined with animation playback rendered in real time via a game engine. Users can interact with the content in real time at their chosen timing and influence what is displayed on the screen. Real-time rendering presents the user with predefined assets generated for a portion of the linear content (e.g., an episode, short, or movie), while the user can interact with the content at any time and see the results of the interaction immediately on the screen.

[0009] In this embodiment, predefined linear content assets can be combined with real-time generated content, using a workflow that enables content creators to seamlessly switch between linear real-time data and interactive events to create a continuous, user-driven experience.

[0010] In one embodiment, this can be achieved by integrating a master linear timeline or a real-time sequencer (e.g., Unity Technologies' Timeline, Unreal Engine's Sequence Editor, or other suitable technology) into an interactive state machine (e.g., an interactive state machine with sequential states). For example, a linear timeline (e.g., a master linear timeline) can be used with a state machine component layered on top of an existing timeline. In an embodiment, this approach allows a content creator to reference predefined linear content, activate and deactivate interactive capabilities based on time, and add or insert branched content, while maintaining the performance intent of the linear content.

[0011] In this embodiment, using a linear timeline allows a content creator's creative team to have strong, repeatable control over the data. For example, the creative team can control appropriate predefined linear content assets, which include camera animation, audio, sound design, character animation, lighting, shading, modeling, other visual effects (VFX), or any other appropriate assets. That is, in this embodiment, a content creator can reference any appropriate assets from the linear content in an interactive element and choose which assets to include.

[0012] Furthermore, the use of a linear timeline allows content creators to remove desired objects from the linear timeline and make them user-interactive, and vice versa. Additionally, the use of a linear timeline enables content creators to navigate to any point in the timeline based on any interactive event or combination of interactive events, without requiring a state machine or procedural implementation. For example, a content creator can navigate to a specific timestamp in the timeline at any given moment based on any user interaction or other trigger.

[0013] In embodiments, one or more techniques described herein differ from conventional video game development, which relies on procedural methods (e.g., state machines) for interactive applications. For example, video game development may refer to the end frame or final frame of a video in a state machine. In video game development, there is generally no linear timeline that progresses independently of user input, and progression between scenes is state-dependent, not time-dependent. Therefore, in conventional video game development, scene transitions were typically presented as "hard cuts," as opposed to seamless transitions.

[0014] One or more of the techniques described herein differ from traditional video game development by referencing linear content assets (e.g., audio data, camera data, animation data, and other appropriate assets) in real time. By using a linear timeline approach and limiting the use of proceduralism for interactive content, it is possible to simultaneously enable user interaction at any point in the content while preserving the content creator's artistic intent regarding audio, camera, and animation. The creative team can easily play back and review any part of the content (by scrubbing through timecode, etc.) as if working with a purely linear version of the project, while maintaining interactivity. Content creators can flexibly add procedural cues that users can invoke, such as executing shader or camera movements, while playing along with the main sequence of the content. For example, the camera may be on a linear trajectory, while additional animations, such as one or more characters moving from one point to another, can be provided as interactive logic.

[0015] Furthermore, in embodiments, the linear timeline approach described herein allows interactive elements to reference linear content (for example, any appropriate predefined asset of linear content, as described later in relation to Figure 1). This makes it possible to develop the linear and interactive aspects of the content in parallel and non-destructively. In this embodiment, upgrades made to linear content assets (e.g., audio, camera, or animation) are automatically updated in the interactive content. This makes it possible to develop content quickly and efficiently. This will be further explained below in relation to Figures 7A and 7B.

[0016] In this embodiment, enabling interactive elements to reference linear content assets further leads to more efficient creation of interactive elements, resulting in reduced memory usage. For example, linear content assets can be mixed and matched in different combinations to create interactive elements for the user. Suppose a character in an interactive digital experience has three assets, each with two possible states: 1) an animation of the character being happy or sad, 2) whether the hat is red or white, and 3) an animation of whether the background animation is exploding or not. These three assets and their respective two possible states can be used to generate eight possible interactive elements (i.e., [happy, red, explosion], [happy, red, no explosion], [happy, white, explosion], [happy, white, no explosion], [sad, red, explosion], [sad, red, no explosion], [sad, white, explosion], or [sad, white, no explosion]). In a conventional solution, all eight of these interactive elements might be stored individually in memory for presentation to the user. Instead, by allowing interactive elements to reference linear content assets, only three assets need to be stored in memory to generate eight potential interactive elements. This significantly reduces memory usage and improves the efficiency of creating interactive elements.

[0017] Figure 1 shows a computing environment 100 for creating an interactive digital experience for linear content according to one embodiment. In one embodiment, an interactive content controller 110 generates interactive content 120 using one or more linear content assets 102. The linear content assets 102 may include predefined assets used for linear content, which include camera animation assets (e.g., virtual camera data or any other suitable camera animation assets), audio assets (e.g., audio files or any other suitable audio assets), sound design assets, animation assets (e.g., character animation, object animation, etc.), lighting assets, shading assets, modeling assets, VFX, or any other suitable assets.

[0018] For example, the content generation service 112 can generate interactive content 120 using the linear content asset 102. The interactive content 120 may include a reference to the linear content asset 102 along with one or more interactive content elements. In this embodiment, the content generation service 112 generates the interactive content 120 by integrating a linear timeline or a real-time sequencer with one or more interactive state machines. For example, the content generation service 112 can use a linear timeline, layer state machine components on the timeline, and generate the interactive content 120 using the linear content asset 102. This will be further explained below in relation to Figures 3 to 5.

[0019] In this embodiment, the interactive content controller 110 provides the interactive content 120 to one or more content development servers 130 (hereinafter referred to as the content development servers 130). For example, the interactive content 120 may be an executable file. The executable file is generated by the interactive content controller 110 (for example, using the content generation service 112), sent to the content development servers 130, and can be maintained in an appropriate repository (for example, the interactive content repository 140). In this embodiment, the content development servers 130, the interactive content repository 140, or both are appropriate cloud computing infrastructure (for example, a public cloud, a private cloud, a hybrid cloud, or any other appropriate cloud computing infrastructure). Alternatively, or additionally, the content development servers 130, the interactive content repository 140, or both are on-premises compute infrastructure (for example, a collection of on-premises servers) or any other appropriate compute infrastructure.

[0020] In this embodiment, the interactive content 120 is provided to one or more user devices 150. In this embodiment, the interactive content executable (e.g., the interactive content 120) can be played back via a content development server 130 (e.g., using a graphics processing unit (GPU)) and the display content (e.g., pixel data) can be streamed to one or more user devices 150 (hereinafter, user devices 150) using a suitable communication network. User devices 150 can include any computing device, which includes smartphones, tablets, laptop computers, desktop computers, televisions (e.g., smart TVs), streaming players, wearable devices, head-mounted displays, or any other suitable computing device. In this embodiment, this enables a fully interactive experience for the user of user device 150 while requiring only a thin client with minimal processing and device capabilities (e.g., the main requirement is a high-speed network connection between user device 150 and content development server 130). This offloads the processing burden from user device 150 and allows for greater flexibility in creative execution.

[0021] Alternatively or additionally, executable interactive content can also be played on the user device 150. For example, the user device 150 can detect local computer capabilities and, based on these capabilities, select remote or local execution. A user device 150 with limited computer capabilities (e.g., computer capabilities that do not meet appropriate thresholds), a user device 150 with high-speed network connectivity (e.g., network capabilities that meet appropriate thresholds), or both, can receive a stream of pixel data from an application running on the content development server 130, while a more capable user device 150, or a user device 150 with more limited network connectivity, can run the application locally. In embodiments, executable interactive content can be provided to the user device 150 using a communication network, via a physical medium (e.g., an appropriate physical disk or drive), or using any other appropriate technology. As another example, interactive elements can be generated via a combination of execution on the client device and a remote server (e.g., some elements generated on the remote server and provided as pixel data, and other elements generated locally).

[0022] Furthermore, in an embodiment, the user device 150 (or the content development server 130) can determine whether to stream a full linear version of the content or an interactive version of the content. For example, if a particular user device has limited network connectivity and limited computer capabilities, the user device (or the content development server) can choose to stream the full linear content. If the user device has sufficient network connectivity or sufficient local computer capabilities, the user device (or the content development server) can choose to execute and stream the interactive content. These techniques are further described in U.S. Patent No. 10,924,823, which is incorporated herein by reference.

[0023] In an embodiment, any or all of the elements of the computing environment 100 (e.g., the linear content asset 102, the interactive content controller 110, the content development server 130, the interactive content repository 140, and the user device 150) are connected using one or more communication networks. For example, the elements of the computing environment 100 can interact using any combination of a local area network (LAN), a wide area network (WAN), the Internet, or any other suitable communication network. Further, the elements of the computing environment 100 can connect to the communication network using any suitable network connection, including a wired connection (e.g., Ethernet or fiber optic connection), a wireless connection (e.g., WiFi connection), a cellular connection, or any other suitable network connection.

[0024] FIG. 2 shows an interactive content controller 110 (e.g., shown in FIG. 1) for creating an interactive digital experience for linear content according to one embodiment. The interactive content controller 110 includes a processor 202, a memory 210, and a network component 220. The memory 210 can take the form of any non-transitory computer-readable medium. Generally, the processor 202 retrieves and executes programming instructions stored in the memory 210. The processor 202 represents a single central processing unit (CPU), multiple CPUs, a single CPU having multiple processing cores, a graphics processing unit (GPU) having multiple execution paths, and the like.

[0025] The network component 200 includes components necessary for the interactive content controller 110 to interface with a suitable communication network (e.g., a communication network interconnecting various component configurations of the computing environment 100 shown in FIG. 1, or a communication network interconnecting the computing environment 100 with other computing systems). For example, the network component 220 can include a wired, WiFi, or cellular network interface component, and associated software. Although the memory 210 is shown as a single entity, the memory 210 can include one or more memory devices having memory blocks associated with physical addresses such as random access memory (RAM), read only memory (ROM), flash memory, or other types of volatile and / or non-volatile memory.

[0026] Generally, memory 210 contains program code for performing various functions related to the use of the interactive content controller 110. Generally, the program code is written as various functional "applications" or "modules" within memory 210, although alternative implementations may have different functions and / or combinations of functions. Within memory 210, the content generation service 112 facilitates the generation of interactive content (for example, generating interactive content using linear content assets by integrating a linear timeline or real-time sequencer with an interactive state machine). This will be discussed further in relation to Figure 3.

[0027] Although the interactive content controller 110 is illustrated as a single entity, in embodiments, various components can be implemented using a physical computing system, cloud compute nodes and storage locations, or any other suitable combination of implementations. For example, the interactive content controller 110 can be implemented using a server or a cluster of servers. For example, one or more components of the interactive content controller 110 can be implemented using a public cloud, private cloud, hybrid cloud, or any other suitable implementation. Furthermore, the interactive content controller 110 may include one or more computing platforms, such as computer servers, which may coexist or form an interactively linked but distributed system, such as a cloud-based system.

[0028] Furthermore, although Figure 2 depicts the content generation service 112 as being located in memory 210, this representation is provided merely as an example for clarity. More generally, the interactive content controller 110 may include one or more computing platforms, such as computer servers, which may reside together or form an interactively linked but distributed system, such as a cloud-based system. As a result, the processor 202 and memory 210 may correspond to distributed processor and memory resources within the computing environment 100. Therefore, it is understood that the content generation service 112 may be stored remotely within the distributed memory resources of the computing environment 100.

[0029] Figure 3 is a flowchart 300 illustrating a method for creating an interactive digital experience for linear content according to one embodiment. In block 302, a content generation service (e.g., content generation service 112 as illustrated in Figures 1-2) identifies a linear content asset. For example, as described above in relation to Figure 1, the content generation service may receive a linear content asset 102. In the embodiment, the linear content asset may include predefined assets used in the linear content, which include camera animation assets (e.g., virtual camera data or any other suitable camera animation assets), audio assets (e.g., audio files or any other suitable audio assets), sound design assets, animation assets (e.g., character animation, object animation, etc.), lighting assets, shading assets, modeling assets, VFX, or any other suitable assets.

[0030] In block 304, the content generation service generates one or more interactive sequences (for example, by a combination of referencing linear content assets and adding additional interactive logic). For example, the content generation service can generate interactive sequences that have interactive logic to control world-space interactions and screen-space interactions (for example, in response to user input or other inputs). This will be further explained below in relation to Figure 4. In embodiments, world-space interactions may include passive and active capabilities, and both world-space and screen-space interactions may use interactive logic to control interactive moments and branched narration in a linear timeline.

[0031] In block 306, the content generation service combines linear content and interactive sequences on a timeline. For example, the content generation service can use a linear timeline to reference both interactive moments (e.g., described using a state machine) and linear content. This will be discussed further in relation to Figures 7A and 7B. In embodiments, the timeline may include branched narration (e.g., including twigging, as discussed in relation to Figure 4). Furthermore, in embodiments, the combined timeline may reference both interactive content elements and linear content assets.

[0032] In block 308, the content generation service provides interactive content to the user. For example, as described above in relation to Figure 1, in this embodiment, the content generation service generates an executable file containing interactive content (e.g., interactive content 120 illustrated in Figure 1). The executable file can be streamed from a suitable content development server (e.g., one or more of the content development servers 130 illustrated in Figure 1) to a user device (e.g., one or more of the user devices 150 illustrated in Figure 1) using a suitable communication network. Alternatively or additionally, the executable file can be transferred to the user device (e.g., using a communication network) and played on the user device. These are merely examples, and any suitable technology can be used.

[0033] Figure 4 is a block diagram illustrating a generated interactive digital experience for linear content according to one embodiment. In this embodiment, one or more inputs 402 trigger interactive logic 404. For example, input 402 may include user input (e.g., as part of an interactive TV experience). Interactive TV is just one embodiment and can use any suitable content. User input may be touch input, input using an electronic device (e.g., smartphone, mouse or keyboard, remote control), or any other suitable user input (e.g., voice input or visual input (e.g., face capture or gesture recognition)). Furthermore, input 402 may include additional inputs other than user input. For example, input 402 may include passively collected data, which may include environmental data (e.g., weather, time of day, location of the user device, or any other suitable environmental data). Input 402 may further include user device data (e.g., orientation or angle of the user device relative to a baseline, type of user device, size of the device display, or any other suitable device data). These are just examples, and input 402 may include any suitable input data.

[0034] In the embodiment, the interactive logic 404 can provide an interactive experience based on the input 402. For example, the interactive logic 404 can provide an interactive element accompanied by linear content. Furthermore, in the embodiment, the interactive logic can include multiple interactive aspects that occur simultaneously. For example, a character (e.g., a character in linear content) and an object (e.g., an object in linear content) can be made available simultaneously for user interaction. Through the input 402, the end user can choose to interact with the character, the object, or both.

[0035] The interactive logic 404 can be coupled to one or more world-space interactions 406, screen-space interactions 408, or both. In embodiments, the world-space interaction 406 is an interaction with the user within the content. For example, the user can control a character in linear content to move around the screen, or interact with an object or other character in a virtual word. The world-space interaction 406 will be further described below in relation to Figure 5.

[0036] In the embodiment, in contrast, screen-space interaction 408 is a user interface interaction with the screen (e.g., a two-dimensional interaction). For example, the user interface may include buttons to allow the user to display a menu or other items on the screen. As another example, the user interface may include a trigger that invokes an interface that allows the user to scan a fingerprint to interact with linear content. These are examples of screen-space interactions. In the embodiment, both world-space interaction 406 and screen-space interaction 408 may include quick-time events (QTEs).

[0037] Returning to World Space Interaction 406, as will be discussed further in relation to Figure 5, World Space Interaction can enable context-aware interactions (e.g., time-aware interactions) between the user and the content. For example, in traditional video games, World Space Interaction (e.g., QTEs) allows the user to perform an action (e.g., using a control device) immediately after an object (e.g., an instruction or prompt) appears on the screen. For example, the user may only have a limited time (e.g., a few seconds) to interact after an instruction or prompt. QTEs are just one example. Interactive logic can be linked to any appropriate game element (e.g., the user can discover interactive logic through interaction with the game environment without QTEs).

[0038] A world-space interaction 406 can connect to one or more passive capabilities 410 (e.g., passive interactive sequences), one or more active capabilities 412 (e.g., active interactive sequences), or both. In embodiments, passive capabilities 410 do not affect the linear timeline and linear content flow. For example, a user can interact with an object on the screen (e.g., highlight an object, change the color of an object, move an object on the screen). This interaction is visible to the user but does not affect the larger storyline of the linear content. This is an example of a passive capability.

[0039] In this embodiment, the active ability 412 can influence the linear timeline and the flow of linear content. For example, the active ability can lead to branching decisions within the linear content. As an example, the user can control a character to make decisions or take actions related to the linear content. These decisions or actions can influence the story of the linear content and potentially lead to branching decisions. This is an example of an active ability.

[0040] In embodiments, the active capability 412 does not require direct user action. For example, the interactive logic 404 may include a counter or timer that tracks user input. If the user interacts sufficiently (e.g., provides sufficient user input within a given period), the linear timeline (e.g., the linear timeline 422, further described below) may branch into a more interactive section (e.g., using the branched narration 428, further described below). If the user does not interact sufficiently (e.g., provides no user input or provides less user input than required to meet a threshold), the linear timeline may branch into a less interactive section. This allows users who provide more active input to have an interactive experience tailored to their preferred experience, while users who prefer a more passive experience can experience the linear content with less required interaction. In embodiments, this is an example of an active capability because it affects the linear timeline (e.g., leading to a branching decision).

[0041] In this embodiment, both screen-space interaction 408 and active capability 412 lead to additional interactive logic 414. Both screen-space interaction 408 and active capability 412 can influence the linear timeline and lead to branching decisions. As will be described later in relation to the linear timeline 422, the additional interactive logic 414 can govern these decisions.

[0042] Figure 4 further includes structural elements 420. In embodiments, the linear timeline 422 interacts with interactive moments 424. For example, an interactive moment 424 can be controlled by one or more state machines 426. For example, a state machine 426 can describe a number of sequential states for the interactive moment 424 (e.g., progression through sequential states based on user interaction). In embodiments, the interactive moments 424 (e.g., controlled by one or more state machines 426) occur in conjunction with the progression of the linear timeline 422. As described above, these interactive moments 424 can be tied to specific parts of the linear timeline 422, but the linear timeline 422 controls the larger experience. Furthermore, as described above, multiple interactive moments can be used simultaneously (e.g., interactive moments related to multiple characters on the screen, multiple objects on the screen, or combinations of characters and objects).

[0043] Furthermore, the linear timeline 422 includes branched narration 428. In embodiments, the branched narration includes twigging (branching). In embodiments, twigging allows the linear timeline 422 to include multiple branches leading to a limited number of endpoints. For example, the branched narration 428 may include multiple branches that, depending on user interaction, return to the linear timeline 422 or lead to one of a limited number of endpoints. For example, the linear timeline 422 may include two possible endings. Twigging ensures that the branched narration 428 offers the user multiple choices, but all choices lead to one of two available endings. In embodiments, twigging provides a more immersive user experience by effectively concealing the results of user interaction.

[0044] Furthermore, in the embodiment, the linear timeline 422 controls the user experience. For example, user input (e.g., input 402) can, in response to the interaction, guide the content to move to different parts of the linear timeline 422 (e.g., forward or backward along the timeline, or between branches within the timeline). The user experience is then governed by the new part of the linear timeline 422.

[0045] Figure 5 further illustrates the use of world-space interaction to create an interactive digital experience for linear content according to one embodiment. In one embodiment, Figure 5 provides further details of the exemplary world-space interaction 406 shown in Figure 4. As described above in relation to Figure 4, input 502 (e.g., input 402 illustrated in Figure 4) leads to interactive logic 504 (e.g., interactive logic 404 illustrated in Figure 4). Interactive logic 504 leads to physics engine 506.

[0046] In this embodiment, the physics engine 506 interacts with the interactive logic 504 to determine the output from the input 502 based on elements in the content (for example, based on raycast collisions in the content). For example, the user can control a character in the content to interact with an object. The physics engine 506 is used to determine how the interaction between the character and the object affects the digital experience (for example, how it affects the visual display and the linear timeline (for example, the linear timeline 422 shown in Figure 4)).

[0047] As an example, suppose an interactive digital experience includes a vacuum mechanism that attempts to quickly clean up particle effects displayed on the screen. In this example, particle effects are displayed on the screen (e.g., a cloud or group of particles). The user touches the screen near the particles. If the user's touch is close enough to the particle's location (e.g., within a threshold distance), the particle flows towards the point of the user's touch (e.g., the particle moves towards the user's touch and appears on the screen). The appearance, feel, and other elements of the particles displayed on the screen respond to the user's input. This is just an example, and any suitable interactive logic 504 and physics engine 506 can be used.

[0048] In the embodiment, the physics engine 506 leads to additional interactive logic 508. In the embodiment, this interactive logic (e.g., interactive logic 504, interactive logic 508, or both) can further influence the timeline branching. For example, as described above in relation to Figure 4, additional interactive logic 414 can influence the branched narration 428 relative to the linear timeline 422. For example, continuing the vacuum example described above, a user touch that is close enough to a particle (e.g., within the threshold described above) removes some of the particles (e.g., removes them from the screen). If the user succeeds in removing a sufficient number of particles, the interactive logic branches in the timeline. This is just an example, and the initial interactive logic 504 can influence the timeline branching. For example, input 502 can influence the timeline branching (e.g., using interactive logic 504).

[0049] Furthermore, in the embodiment, additional interactive logic 508 leads to a visual effect feature 510 (e.g., an animated character mesh). In the embodiment, the animated character mesh is a visual effect feature that references character animation to display unique visual effects in response to character movement and user interaction. Following the vacuum example described above, user touch on the screen can affect the character mesh of a character appearing on the screen (e.g., bending the character mesh toward or away from the user's touch). This is just one example of a visual effect feature 510. Moreover, the visual effect feature 510 is optional. As will be discussed later, the illustrated order of the blocks in the flowchart is merely an example and not limiting.

[0050] In this embodiment, the visual effect feature 510 leads to a particle system 512 that governs the movement of particles 514. For example, continuing the vacuum example again, the particle system 512 can control the visual display of particles being removed by user touch. Particles are merely an example, and any suitable visual effect can be controlled or used. Furthermore, in this embodiment, the particle system 512 and particle movement 514 can lead to other interactive logic. For example, this additional interactive logic (or any other suitable interactive logic) can control the speed at which the timeline progresses (e.g., half speed, double speed, etc.).

[0051] Figure 6 is a flowchart of a timeline for creating an interactive digital experience for linear content according to one embodiment. In this embodiment, the flow begins at a linear passive decision point 602. As described above, in this embodiment, the passive decision point 602 can enable user interaction (for example, using the passive capability 410 illustrated in Figure 4), but the user interaction does not affect the flow of the linear content (for example, the flow continues to the interactive sequence 604 regardless of user interaction).

[0052] Next, the flow proceeds to interactive sequence 604. Interactive sequence 604, in contrast to the linear passive decision point 602, allows for user interaction that can influence the linear flow (e.g., using the active capability 412 illustrated in Figure 4). For example, interactive sequence 604 leads to interactive decision point 606. Interactive decision point 606 branches in two possible directions. This is just one example, and an interactive decision point can lead to any appropriate number of branches (e.g., three or more branches).

[0053] The first branch includes a linear sequence having interactive capabilities 608 and a resulting passive sequence 610. In the embodiment, the interactive linear sequence 608 is a sequence of linear content (e.g., non-branching content) in which the user can interact with the content without affecting the flow of content. In the embodiment, the passive sequence 610 is a passive sequence of linear content in which the user does not interact with the content.

[0054] The second branch includes another interactive sequence 612 and the resulting passive sequence 614. For example, a user action in interactive sequence 604 influences the branch taken at interactive decision point 606, leading the flow to blocks 612 and 614 instead of blocks 608 and 610. In this embodiment, interactive sequence 612 provides the user with content and interaction without affecting the content flow. In this embodiment, passive sequence 614 (similar to passive sequence 610 described above) is a passive sequence of linear content in which the user does not interact with the content.

[0055] Different paths recombine at interactive decision point 616. As mentioned above in relation to the branched narration 428 illustrated in Figure 4, this is an example of twigging. Interactive decision point 606 leads to two branches, each with different content, but the branches recombine at interactive decision point 616. For example, a linear sequence with interactive capability 608 and the resulting passive sequence 610 can provide different content to the user compared to an interactive sequence 612 and the resulting passive sequence 614, but both ultimately arrive at the same interactive decision point 616.

[0056] Subsequently, the interactive decision point 616 branches into two possible directions. This is again merely an example, and the interactive decision point 616 can branch into any appropriate number of directions (e.g., three or more directions). The first branch includes an interactive sequence 618, a linear sequence 620 with interactive capabilities, another interactive sequence 622, and another linear sequence 624 with interactive capabilities. In embodiments, the interactive sequences 618 and 622 may each include a state machine (e.g., state machine 426 illustrated in Figure 4) in which the user repeats the sequence until a specific interaction is provided or a goal is reached. For example, the user may remain in the interactive sequence 618 while attempting to achieve a goal, and then transition to the linear sequence with interactive capabilities 620. These are merely examples, and the interactive sequences 622 and the linear sequence 624 with interactive capabilities are optional and can be omitted. Alternatively, the flow may include only the interactive sequence 618 and the linear sequence 620 with interactive capabilities, or it may include additional interactive sequences and linear sequences with interactive capabilities, or it may include any appropriate content elements.

[0057] The second branch includes a passive sequence 626 (e.g., a sequence without user interaction, as described above) and avoids interactive elements 618 and 622. For example, the interactive decision point 616 may include logic to track the user's previous level of interactivity. If the user has reached a sufficient threshold of interactivity (e.g., a sufficient number of interactions in a given time period), the user is sent to the branch with interactive elements 618 and 622. If the user has not reached a sufficient threshold of interactivity, the user is assumed to be interested in a more passive linear experience and is sent to the branch with the resulting passive sequence 626. The branches reconnect at the interactive decision point 628.

[0058] Next, the interactive decision point 628 leads to two branches. The first branch includes a linear sequence and a passive sequence 632 having passive capabilities 630. This leads to a linear, passive decision point 634. The second branch includes an interactive sequence 636 and a passive sequence 638. This leads to a linear-passive decision point 640. In this embodiment, the linear-passive decision points 634 and 640 represent alternative endings for the interactive content. This is merely an example, and the linear-passive decision points 634 and 640 can represent any suitable content.

[0059] Figures 7A-7B illustrate a reference between linear time and interactive timelines for creating an interactive digital experience for linear content according to one embodiment. In one embodiment, the interactive timeline 730 illustrated in Figure 7B references both the linear timeline portion 732 and the interactive timeline portion 734. For example, the linear timeline portion 732 illustrated in Figure 7B may reference the linear timeline 710 illustrated in Figure 7A. This allows the interactive timeline 730 to reference the linear content assets contained in the linear timeline 710. These linear content assets contained in the linear timeline 710 may include lighting data 712, audio data 714, linear elements 716A-716E (e.g., title card data, heads-up display (HUD) data, text data, or any other suitable linear elements), effect data 718, animation data 720, and any other suitable content assets.

[0060] In this embodiment, the interactive timeline 730 can reference these linear content assets from the linear timeline 710, thereby enabling the propagation of changes made to the linear content timeline 710 to the interactive timeline 730. Furthermore, the interactive timeline 730 may include interactive elements 742A-742C for presentation between interactive timeline segments. As described above, this ability to reference the linear timeline from the interactive timeline enables parallel development of the linear timeline 710 and the interactive timeline 730, significantly reducing development costs and potential errors.

[0061] This disclosure refers to various embodiments. However, it should be understood that this disclosure is not limited to any particular described embodiment. Rather, any combination of the following features and elements, whether related to a different embodiment or not, is considered an implementation and practice of the disclosure provided herein. Furthermore, where an element of an embodiment is described in the form of “at least one of A and B,” it is understood that embodiments exclusively including element A, exclusively including element B, and embodiments including elements A and B are intended, respectively. Furthermore, some embodiments may achieve advantages that exceed other possible solutions or the prior art, but whether a particular advantage is achieved by a given embodiment does not limit this disclosure. Accordingly, the aspects, features, embodiments and advantages disclosed herein are merely illustrative and should not be considered elements or limitations of the appended claims unless expressly stated in the claims. Similarly, references to “invention” should not be interpreted as a generalization of any inventive subject matter disclosed herein and should not be considered elements or limitations of the appended claims unless expressly stated in the claims.

[0062] As those skilled in the art will understand, the embodiments described herein can be embodied as systems, methods, or computer program products. Accordingly, embodiments can take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware aspects, all of which may be generally referred to herein as “circuits,” “modules,” or “systems.” Furthermore, embodiments described herein can take the form of computer program products embodied in one or more computer-readable media having embodied computer-readable program code.

[0063] Program code embodied on a computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination thereof.

[0064] Computer program code for performing operations for embodiments of the present disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and traditional procedural programming languages ​​such as the "C" programming language, or similar programming languages. The program code can be fully executed on the user's computer, partially executed on the user's computer, or fully executed as a standalone software package on the user's computer, partially on a remote computer, or on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer via any type of network (e.g., including a local area network (LAN) or a wide area network (WAN)), or the connection may be made to an external computer (e.g., via the Internet using an Internet service provider).

[0065] Aspects of the present disclosure are described herein with reference to flowcharts or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. Each block in a flowchart or block diagram, and any combination of blocks in a flowchart or block diagram, is understood to be executable by computer program instructions. These computer program instructions are provided to a processor of a general-purpose computer, a special-purpose computer, or any other programmable data processing device that forms a machine, thereby enabling instructions executed through the processor of the computer or other programmable data processing device to form means for performing the actions / operations specified in the blocks of the flowchart or block diagram.

[0066] Furthermore, these computer program instructions can be stored in a computer-readable medium that can instruct a computer, other programmable data processing device, or other device to function in a specific way, thereby enabling the instructions stored in the computer-readable medium to generate a manufactured article containing instructions that perform functions / actions specified in a flowchart or block diagram.

[0067] Computer program instructions are loaded into a computer, another programmable data processing device, or other device, and a series of operational steps are executed on the computer, other programmable data processing device, or other device to form a computer execution process, thereby providing a process for instructions executed on the computer, other programmable device, or other device to perform functions / actions identified in blocks of a flowchart or block diagram.

[0068] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for performing a particular logic function. It should also be noted that in some alternative implementations, the functions described in a block may be executed in a different order than shown in the diagram. For example, two blocks shown consecutively may actually be executed substantially simultaneously, and blocks may be executed in reverse order or out of order depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and any combination of blocks in a block diagram or flowchart, can be executed by a special-purpose hardware-based system or a combination of special-purpose hardware and computer instructions to perform a specified function or action.

[0069] While the foregoing applies to embodiments of the present disclosure, other and further embodiments of the present disclosure may be created without departing from its basic scope, the scope of which is defined by the following claims.

Claims

1. The process of identifying multiple assets related to the presentation of linear content, A process for generating interactive content using multiple linear content assets, A process for generating an interactive sequence that references one or more of multiple assets, wherein the interactive sequence includes a state machine containing multiple sequential states, and the progression through the multiple sequential states is based on user interaction. A process for combining linear content and an interactive sequence on a timeline that sequentially describes linear content, wherein the timeline includes one or more branches related to the linear content, the selection of a first branch of the one or more branches is initiated by user input during the interactive sequence, and the selection of the first branch generates the interactive content through a process that affects the story of the linear content. A method that includes the step of sending interactive content to a user.

2. The method according to claim 1, wherein the step of generating an interactive sequence that references one or more of a plurality of assets includes at least one of (i) directly referencing one or more of a plurality of assets, or (ii) referencing linear content related to one or more of a plurality of assets.

3. The method according to claim 1, wherein the selection of a first branch includes moving to a selected location in the timeline.

4. The method according to claim 1, wherein the step of generating an interactive sequence includes controlling passive capabilities, and the step of generating interactive content using linear content includes generating an additional interactive sequence having interactive logic that controls active capabilities related to the linear content.

5. The method according to claim 4, wherein controlling passive capabilities includes changing what is displayed on a first display of at least a portion of linear content for the user without affecting the progression through the timeline, and an additional interactive sequence having interactive logic for controlling active capabilities changes what is displayed on a second display of at least a portion of linear content for the user and affects the progression through the timeline.

6. The method according to claim 1, wherein the linear content comprises at least one film or episode.

7. The method according to claim 1, wherein the multiple assets include at least one of camera animation, audio, sound design, character animation, lighting, shading, modeling, or visual effects (VFX).

8. The method according to claim 1, wherein the timeline includes multiple branches related to linear content, and the multiple branches depend on an interactive sequence.

9. A non-temporary computer-readable recording medium containing computer program code that performs an operation when executed by one or more computer processors, wherein the operation is: A process for generating interactive content using linear content, A process for generating an interactive sequence that references one or more of multiple assets, wherein the interactive sequence includes a state machine containing multiple sequential states, and the progression through the multiple sequential states is based on user interaction. A process for combining linear content and an interactive sequence on a timeline that sequentially describes linear content, wherein the timeline includes one or more branches related to the linear content, the selection of a first branch of the one or more branches is initiated by user input during the interactive sequence, and the selection of the first branch generates the interactive content through a process that affects the story of the linear content. A computer-readable recording medium that includes a process for transmitting interactive content to a user.

10. The non-temporary computer-readable recording medium according to claim 9, wherein the selection of a first branch includes moving to a selected location in the timeline.

11. The non-temporary computer-readable recording medium according to claim 9, wherein the step of generating an interactive sequence includes controlling passive capabilities, and the step of generating interactive content using multiple assets of linear content includes generating an additional interactive sequence having interactive logic that controls active capabilities associated with the linear content.

12. A non-temporary computer-readable recording medium according to claim 11, wherein controlling passive capabilities changes what is displayed on a first display of at least a portion of linear content for the user without affecting the progression through the timeline, and an additional interactive sequence having interactive logic to control active capabilities changes what is displayed on a second display of at least a portion of linear content for the user and affects the progression through the timeline.

13. It is a system, Computer processor and When executed on a computer processor, it is memory that stores instructions for performing an operation, and the operation is, The process of identifying multiple assets related to the presentation of linear content, A process for generating interactive content using multiple linear content assets, A process for generating an interactive sequence that references one or more of multiple assets, wherein the interactive sequence includes a state machine containing multiple sequential states, and the progression through the multiple sequential states is based on user interaction. A process for combining linear content and an interactive sequence on a timeline that sequentially describes linear content, wherein the timeline includes one or more branches related to the linear content, the selection of a first branch of the one or more branches is initiated by user input during the interactive sequence, and the selection of the first branch generates the interactive content through a process that affects the story of the linear content. A system with memory that includes a process for sending interactive content to the user.

14. The system according to claim 13, wherein the selection of a first branch includes moving to a selected location in the timeline.

15. The method according to claim 1, wherein the linear content is combined with an interactive sequence, and this is performed using a workflow that enables seamless switching between linear real-time data and interactive events.

16. The non-temporary computer-readable recording medium according to claim 9, wherein the combination of linear content with an interactive sequence is performed using a workflow that enables seamless switching between linear real-time data and interactive events.

17. The system according to claim 13, wherein the combination of linear content with an interactive sequence is performed using a workflow that enables seamless switching between linear real-time data and interactive events.

Images