SYSTEM AND METHOD FOR INSERTING SUPPLEMENTAL CONTENT INTO CLOUD-RENDERED MEDIA CONTENT - Patent application

Abstract

Disclosed herein are systems and methods for inserting supplemental content into cloud-rendered media content, such as video game cut scenes. The media content is rendered and encoded for streaming to a client device. Data related to a current session associated with the client device is retrieved. Based on the retrieved session data, supplemental content items are identified for insertion into the streaming media content. When an insertion point is detected in the streaming media content, the supplemental content items are streamed to the client device in place of the streaming media content.

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The present disclosure is directed to cloud-rendered streaming media content. In particular, techniques for the insertion of supplemental content into cloud-rendered streaming media content are disclosed. Summary of the Invention [Means for solving the problem]

[0002] Online purchase of games with the ability to download, install and run the games locally has been around for a long time. Companies are now beginning to offer services for users to subscribe and play games that run within a network. This is called cloud gaming. Cloud gaming uses a technique called remote rendering.

[0003] Cloud gaming is gaining support among game studios, internet companies and network operators. Google's Stadia offers remote game streaming as a business. TM , Nvidia's GeForce Now TM , and Sony's PS Now TMThere are products on the market today such as NVIDIA GeForce GTX 1080p, ...

[0004] Cutscenes are included by game studios to provide a video cinematic experience that enhances the game experience and hides long loading times when transitioning from one main gameplay level / world to the next. Game level loading times can be as much as 30-50 seconds when transitioning from one level to the next. Rather than showing a busy cursor (e.g., an animated icon), the game engine provides the capability to render cutscenes, which involve computer animated in-game characters acting out scenes that relate to the gameplay and tell the storyline for the upcoming level. These cutscenes can be up to 5-10 minutes long. The game engine will allow cutscenes to be avoided once the next game level is loaded, allowing the user to continue immediately after the required loading time has elapsed.

[0005] Many newer game cutscenes are now interactive and customized with in-game purchases and upgrades. When the game is running locally and the cutscenes are generated from the same game engine, this is essentially an extension of the game, providing basic interactivity and it is easy to generate custom cutscenes based on the player's character customization. It is important that cloud gaming be able to provide the same experience without consuming the managed network resources and bandwidth required to run the game in its primary gaming mode.

[0006] For most games today, video-based supplemental content, such as advertisements, is not played in the game engine. These supplemental content items are not played when the game is running locally on a PC or game console, nor are they played when they are rendered remotely in the cloud. Generally, supplemental content exists in locally running mobile games. However, it is mostly in the form of pop-up content overlaid on parts of the game, and is not traditional video-based supplemental content. In the mobile gaming world, this is starting to change. For example, Unity recently provided a way for video advertisements to be played in the game. However, in this case, the game is running locally on the mobile device.

[0007] In addition to the supplemental content, there is basic demographic information to target users with in-game supplemental content. These types of supplemental content are usually advertisements and focus on promoting certain apps in app stores such as Google Play or the Apple App Store. Typically, these are advertisements for other games from the same game studio or game studio partners. Game studios also advertise for games or character upgrades that must be purchased using real currency, rather than in-game rewards earned by completing game levels or collecting items in the game. This leads to very limited in-game advertising, especially for video-based ads.

[0008] Disclosed herein are systems and methods for inserting supplemental content into cloud-rendered media content, such as video game cut scenes. The media content is rendered and encoded for streaming to a client device. Data related to a current session associated with the client device is retrieved. Based on the retrieved session data, a supplemental content item is identified for insertion into the streaming media content. For example, game data may be retrieved for a current game session and used to determine which of multiple available supplemental content items would best suit a user of the client device. When an insertion point is detected in the streaming media content, the supplemental content item is streamed to the client device in place of the streaming media content. When streaming of the supplemental content item ends, either at the end of the supplemental content item or in response to a request from the client device to skip the supplemental content item, streaming of the streaming media content is resumed.

[0009] The streaming media content and the supplemental content may be initially received as raw multimedia data, which is then input into a streaming media encoder. The raw multimedia data of the streaming media content may be encoded into a streaming media format and streamed to a client device. When an insertion point is detected during the encoding process, the input of the raw multimedia data of the streaming media content is paused and the raw multimedia data of the supplemental content item is input into the streaming media encoder instead. When it is time to resume streaming of the streaming media content, the input of the raw multimedia data of the supplemental content into the encoder is stopped and the input of the raw multimedia data of the streaming media content is resumed.

[0010] To identify supplemental content for insertion into the streaming media content, a parameter associated with the streaming media content may be read and compared to a reference value. For example, if the streaming media content is a video game cut scene, a parameter may be read from the game data and its value compared to the reference value. A supplemental content item may be selected based on a comparison of the two values. In some cases, the parameter describes a network condition. For example, it may describe the amount of bandwidth available to a client device, or the maximum transmission rate (e.g., 30 Mbps) of the connection through which the client device communicates with a cloud-based media server. In these cases, the reference value may describe a threshold level for the relevant network condition at which a user of the client device would experience better media content performance. If the value of the parameter does not meet or exceed the reference value, a supplemental content item associated with the relevant network condition may be identified for insertion. For example, if the parameter describes a maximum transmission rate with a value of 30 Mbps and the reference value is 50 Mbps, an advertisement for an Internet service provider offering connection speeds above 50 Mbps may be identified for insertion into the video game cut scene.

[0011] In some cases where the streaming media content is a video game cutscene, the parameter describes an in-game condition. For example, it may describe a characteristic or ability level of the user's in-game character. In these cases, the reference value may describe a first ability level. If the value of the parameter meets or exceeds the reference value, a supplemental content item associated with a second ability level higher than the first ability level may be identified for insertion into the video game cutscene. For example, an in-game offer to increase the in-game character's total hit points, stamina, agility, or other characteristic may be identified. If the value of the parameter does not meet or exceed the reference value, a supplemental content item associated with the first ability level may be identified for insertion. For example, an in-game offer to increase the in-game character's statistics to reach the first ability level may be identified for insertion.

[0012] A video game cutscene may be an interactive cutscene with two or more paths that a user may choose at one or more interaction points. Supplemental content items may therefore be identified for insertion into the video game cutscene by first identifying multiple interactive paths within the interactive cutscene. Based on the game data, it may be determined whether a first interactive path has been selected. If so, a supplemental content item associated with the first interactive path is identified for insertion. [Brief description of the drawings]

[0013] These and other objects and advantages of the present disclosure will become apparent upon consideration of the following detailed description taken in conjunction with the accompanying drawings.

[0014] [Figure 1] FIG. 1 is a diagram illustrating the delivery of streaming media content and supplemental content to client devices according to some embodiments of the present disclosure.

[0015] [Diagram 2] FIG. 2 is a diagram illustrating encoding streaming media content and supplemental content into a single output stream according to some embodiments of the present disclosure.

[0016] [Diagram 3] FIG. 3 is a diagram illustrating the insertion at an encoder of supplemental content into streaming media content at an insertion point to generate a single output stream according to some embodiments of the present disclosure.

[0017] [Figure 4] FIG. 4 is a block diagram illustrating components of, and data flows between, a system for inserting supplemental content into cloud-rendered streaming media content according to some embodiments of the disclosure.

[0018] [Diagram 5] FIG. 5 is a flowchart illustrating a process for inserting supplemental content into a cloud-rendered video game cut-scene according to some embodiments of the present disclosure.

[0019] [Figure 6] FIG. 6 is a flow chart illustrating a process for generating a single output stream from a streaming media encoder according to some embodiments of the present disclosure.

[0020] [Figure 7] FIG. 7 is a flowchart depicting a process for identifying and selecting supplemental content items for insertion into a video game cut scene according to some embodiments of the present disclosure.

[0021] [Figure 8] FIG. 8 is a flowchart illustrating a process for identifying supplemental content items for insertion into a cloud-rendered interactive video game cut-scene according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] FIG. 1 is a schematic diagram illustrating the delivery of streaming media content and supplemental content to client devices according to some embodiments of the present disclosure. A cloud-based media server 100 is responsible for ingesting and encoding media content for delivery to client devices. The media server 100 receives media content from a media content source 102 and supplemental content from a supplemental content source 104. The media server 100 encodes the media content and supplemental content in a streaming format. The media server 100 then delivers (106) the media content to a client device 108. When the supplemental content is to be inserted into a presentation of the media content, the media server 100 delivers (110) the supplemental content to the client device 108. In some cases, the media server 100 may use separate streams and even separate encoders to separately encode and deliver the media content and supplemental content. However, in other cases, the media content and supplemental content are encoded into a single output stream by a single encoder. Thus, the ingestion of the supplemental content by the encoder may occur on demand when a supplemental content insertion point is detected in the media content by the encoder.

[0023] 2 is a schematic diagram illustrating encoding of streaming media content and supplemental content into a single output stream according to some embodiments of the present disclosure. A streaming media encoder 200 receives raw media data for media content from a media content source 204 (202). The streaming media encoder encodes the raw media data in a format suitable for streaming, such as H.264, and streams the encoded media content to a client device (206). When supplemental content is to be inserted, the streaming media encoder 200 requests a supplemental content item from a supplemental content source 210 (208). The streaming media encoder 200 receives the raw media data of the requested supplemental content item (212) and encodes it into a streaming format as part of the same output stream as the media content. Thus, the output stream sent to the client device (206) includes the media content and the supplemental content in a linear fashion.

[0024] 3 is a schematic diagram illustrating the insertion of supplemental content in an encoder into streaming media content at an insertion point to generate a single output stream according to some embodiments of the present disclosure. An encoder 300, such as the streaming media encoder 200, receives a raw media stream 302 for media content. The media content includes an insertion point 304. The encoder 300 encodes a first portion 302a of the media content in a streaming format. When the encoder 300 reaches the insertion point 304, the encoder 300 accesses the supplemental content 306. The encoder 300 encodes the supplemental content 306 in a streaming format. Upon completion, the encoder 300 resumes encoding the next portion 302b of the media content. The resulting stream 308 thus consists of the portion 302a followed by the supplemental content 306 and the portion 302b in a single stream.

[0025] 4 is a block diagram illustrating components of, and data flows between, a system for inserting supplemental content into cloud-rendered streaming media content according to some embodiments of the present disclosure. Although the system illustrated in FIG. 4 relates to video game cut scenes, it will be understood by those skilled in the art that the actions described below may be applied to any type of media in which supplemental content items may be inserted.

[0026] The edge server 400 may be a network node in a cloud computing environment, such as a mobile broadband network, and is responsible for encoding and delivering media content to client devices. The edge server 400 includes control circuitry 402. The control circuitry 402 may be based on any suitable processing circuitry and may include control circuitry and memory circuitry, which may be located on a single integrated circuit or may be separate components. As referred to herein, processing circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc., and may include multi-core processors (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores). In some embodiments, the processing circuitry may be distributed across multiple separate processors or processing units, for example multiple processing units of the same type (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor).

[0027] The control circuitry 402 may receive raw media data for video game cut scenes from a game data source 406 (404) or may receive raw video data for video game cut scenes from a dedicated game cut scene source 410 (408). The video game cut scenes are received using transceiver circuitry 412. The transceiver circuitry 412 may comprise a network connection, such as an Ethernet connection, a Wi-Fi connection, a mobile broadband interface, or a connection employing any other suitable networking protocol, and data may be transmitted to and received from a remote device via the network connection. The transceiver circuitry 412 then transmits the received raw media data to streaming media encoding circuitry 416 (414). The streaming media encoding circuitry 416 encodes the raw media data into a suitable multimedia format for streaming using a streaming protocol, such as MPEG-DASH, and packages the converted multimedia data.

[0028] While encoding the raw media data of the video game cutscene, the streaming media encoding circuitry 416 monitors the media data for supplemental content insertion points. For example, the raw media data may include metadata for one or more frames, the metadata indicating the frame at which the supplemental content should be inserted. Once detected, the streaming media encoding circuitry 416 transmits (418) a request for the supplemental content to the supplemental content selection circuitry 420. In some cases, the supplemental content selection circuitry may simply select a paid advertisement, or other scheduled or paid supplemental content, for insertion into the video game cutscene. In other cases, however, the supplemental content selection circuitry 420 may select the supplemental content item based on in-game conditions or network conditions experienced by the client device.

[0029] If a supplemental content item is to be selected based on an in-game condition, the supplemental content selection circuitry 420 transmits (422) a request to memory 424 to retrieve from game session data 426 a parameter of the current game session that is associated with the in-game condition. For example, the parameter may relate to a current ability level, experience level, hit point level, stamina, agility, or other characteristic of the user's in-game character. The supplemental content selection circuitry 420 receives (428) the parameter and compares the value of the parameter to a reference value. For example, the parameter may relate to a current ability level in a game where an increase in that ability level occurs upon accumulation of a set number of points. The supplemental content selection circuitry 420 compares the value of the parameter, which may be a number of points accumulated, to a reference value, which may be a number of points required to reach a particular ability level. If the value of the parameter does not meet or exceed the reference value, the supplemental content selection circuitry 420 may determine that supplemental content that will earn the user additional points should be selected. If the value of the parameter meets or exceeds the reference value, supplemental content associated with other in-game offers may be selected.

[0030] If the supplemental content item is to be selected based on network conditions, the supplemental content selection circuitry 420 transmits (430) a request for network condition information to the network condition analysis circuitry 432. The game session data 426 may include network information data reported by the client device. The network condition analysis circuitry 432 may transmit (434) a request to the memory 424 to read the network information data from the game session data 426. The network information data reported by the client device may include the type of network over which the client device is connected to the edge server 400, the maximum transmission rate observed by the client device, or other information related to the connection between the client device and the edge server 400. The network condition analysis circuitry 432 receives (436) the requested network information data from the game session data 426. The network condition analysis circuitry 432 may alternatively or additionally perform tests on the connection between the client device and the edge server 400. The network condition analysis circuitry 432 may transmit (438) a test packet or "ping" to the client device via the transceiver circuitry 412. The network condition analysis circuitry 432 may record (440) the time the test packet was transmitted and the time an acknowledgment (ACK) message was received from the client device. From these two times and the size of the test packet, the network condition analysis circuitry 432 determines the transmission rate of the connection between the edge server 400 and the client device.

[0031] Once the network conditions are determined, the network condition analysis circuitry 432 transmits a parameter associated with the network condition to the supplemental content selection circuitry 420 (442). The supplemental content selection circuitry 420 compares the value of the parameter to a reference value. For example, the parameter may describe a maximum transmission speed of a connection between the edge server 400 and the client device. The reference value may be a minimum transmission speed required to experience no lag during game play. If the value of the parameter does not meet or exceed the reference value, the supplemental content selection circuitry may determine that supplemental content associated with an increased network transmission speed should be selected.

[0032] Once the supplemental content to be inserted is identified, the supplemental content selection circuitry 420 transmits (444) a request for the identified supplemental content to the transceiver circuitry 412, which in turn transmits (446) the request to the supplemental content source 448. In response, the supplemental content source 448 transmits (450) raw media data for the supplemental content items that match the request to the edge server 400. The raw media data for the supplemental content items is received using the transceiver circuitry 412, which transmits (452) the raw media data to the streaming media encoding circuitry 416.

[0033] The streaming media encoding circuitry 416 pauses capture of the video game cut scene raw media data and instead captures the supplemental content raw media data. Upon completion of the supplemental content or in response to an input from the client device to skip a portion of the supplemental content, the streaming media encoding circuitry 416 resumes capture of the video game cut scene raw media data. The streaming media encoding circuitry 416 continuously outputs the encoded media data as a single media stream (454). The transceiver circuitry 412 transmits the media stream to the client device (456).

[0034] 5 is a flowchart that depicts an illustrative process 500 for inserting supplemental content into a cloud-rendered video game cutscene. Process 500 may be implemented on control circuitry 402. In addition, one or more actions of process 500 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.

[0035] At 502, control circuitry 402 streams a video game cut scene to a client device. Control circuitry 402 encodes raw media data of the video game cut scene into a streaming media format for streaming to the client device. At 504, control circuitry 402 retrieves game data related to a game session associated with the client device. A unique session ID may be associated with the client device. Control circuitry 402 may use the session ID to retrieve game session data for that session. At 506, control circuitry 402 identifies supplemental content items for insertion into the video game cut scene based on the game data. A method for accomplishing this is described below in connection with FIG. 7.

[0036] At 508, the control circuitry 402 monitors the video game cut scene data for a supplemental content insertion point. For example, the video game cut scene data may include a flag or other metadata indication of where the insertion point occurs. At 510, the control circuitry 402 determines whether an insertion point has been detected. If not ("No" at 510), the control circuitry 402 continues to monitor the video game cut scene data at 508.

[0037] If an insertion point is detected ("Yes" at 510), then at 512 the control circuitry 402 streams the supplemental content to the client device in place of the video game cut scenes. This may be accomplished by pausing the streaming of the video game cut scenes and starting a second stream containing the supplemental content. This may also be accomplished by using the method described below in connection with FIG. 6, in which the control circuitry 402 generates a single media stream from both the video game cut scenes and the supplemental content.

[0038] At 514, the control circuitry 402 determines whether an indication has been received that streaming of the supplemental content to the client device has ended. For example, the supplemental content may have ended. Alternatively, an input may be received from the client device corresponding to a command to skip a portion of the supplemental content. If no indication has been received ("No" at 514), the control circuitry 402 continues streaming the supplemental content at 512. If an indication has been received ("Yes" at 514), the control circuitry 402 resumes streaming the video game cut scenes to the client device at 516.

[0039] The actions or descriptions of Figure 5 may be used with any other embodiment of the present disclosure. In addition, the actions and descriptions described in connection with Figure 5 may be performed in alternative orders or in parallel as suitable to further the objectives of the present disclosure.

[0040] 6 is a flowchart depicting an illustrative process 600 for generating a single output stream from a streaming media encoder. Process 600 may be implemented on control circuitry 402. In addition, one or more actions of process 600 may be incorporated within or combined with one or more actions of any other process or embodiment described herein.

[0041] At 602, control circuitry 402 inputs the raw multimedia data of the video game cut scene into a multimedia encoder for encoding in a streaming media format. This may be accomplished using any known streaming media encoding method involving any known streaming media format. During the encoding process, as in FIG. 5 above, control circuitry 402 may monitor the video game cut scene data for a supplemental content insertion point. At 604, control circuitry 402 determines whether an insertion point has been detected. If not ("No" at 604), control circuitry 402 continues to input the raw multimedia data of the video game cut scene into the multimedia encoder at 602.

[0042] If an insertion point is detected ("yes" at 604), the control circuitry 402 pauses the input of the video game cut scene raw multimedia data to the multimedia encoder at 606 and inputs the supplemental content raw multimedia data to the multimedia encoder for encoding in a streaming format at 608. As in FIG. 5 above, the control circuitry 402 may determine whether an indication has been received at 610 that the streaming of the supplemental content to the client device has ended. If not ("no" at 610), the control circuitry 402 continues inputting the supplemental content raw multimedia data to the multimedia encoder at 608. If an indication has been received ("yes" at 610), the control circuitry 402 stops inputting the supplemental content raw multimedia data to the multimedia encoder at 612 and resumes inputting the video game cut scene raw multimedia data to the multimedia encoder at 614.

[0043] The actions or descriptions of Figure 6 may be used with any other embodiment of the present disclosure. In addition, the actions and descriptions described in connection with Figure 6 may be performed in alternative orders or in parallel as suitable to further the objectives of the present disclosure.

[0044] 7 is a flowchart depicting an illustrative process 700 for identifying and selecting supplemental content items for insertion into a video game cut scene. Process 700 may be implemented on control circuitry 402. In addition, one or more actions of process 700 may be incorporated within or combined with one or more actions of any other process or embodiment described herein.

[0045] At 702, control circuitry 402 reads a parameter from the game data. The parameter may describe an in-game condition or a network condition. If the parameter describes a network condition, such as a connection speed, a data transmission rate, a type of network, etc., processing proceeds to the branch beginning at 704. At 706, control circuitry 402 reads a reference value describing a threshold level for the network condition described by the parameter. For example, if the parameter describes a data transmission rate of a connection with a client device, control circuitry 402 may read a reference value for the data transmission rate. The parameter may have a value of 30 Mbps, while the reference value may be 50 Mbps. At 708, control circuitry 402 determines whether the value of the parameter meets or exceeds the reference value. If so, there is no need to present supplemental content (e.g., an advertisement) related to the data transmission rate, and processing returns to 702. If the value of the parameter does not meet or exceed the criteria value (“no” at 708), then at 710 the control circuitry 402 identifies supplemental content items relevant to the network conditions described by the parameter. For example, if the parameter describes a data transmission rate, the control circuitry 402 may identify an advertisement from an Internet service provider or mobile network operator for a faster Internet connection as the supplemental content item.

[0046] If the parameter describes an in-game condition, such as a character statistic, experience level, etc., processing proceeds to a branch beginning at 712. At 714, control circuitry 402 retrieves a criterion value describing a first in-game performance level. For example, the criterion value may be a number of experience points (XP) required to reach the first performance level. At 716, control circuitry 402 determines whether the value of the parameter meets or exceeds the criterion value. For example, control circuitry 402 compares the user's XP to the XP required to reach the first performance level. If the value of the parameter meets or exceeds the criterion value ("yes" at 716), then at 718, control circuitry 402 identifies a supplemental content item associated with a second performance level that is higher than the first performance level. If the value of the parameter does not meet or exceed the criterion value ("no" at 716), then at 720, control circuitry 402 identifies a supplemental content item associated with the first performance level.

[0047] After identifying the supplemental content item associated with the network condition at 710, or the supplemental content item associated with the second capability level at 718, or the supplemental content item associated with the first capability level at 720, the control circuitry selects the identified supplemental content item at 722. The control circuitry 402 may transmit a request for the identified supplemental content item to a supplemental content source. In some embodiments, the control circuitry 402 identifies a type or class of supplemental content item or one or more keywords for the supplemental content item. This data is then used to query the supplemental content source.

[0048] The actions or descriptions of Figure 7 may be used with any other embodiment of the present disclosure. In addition, the actions and descriptions described in connection with Figure 7 may be performed in alternative orders or in parallel as suitable to further the objectives of the present disclosure.

[0049] 8 is a flowchart that depicts an illustrative process 800 for identifying supplemental content items for insertion into a cloud-rendered interactive video game cutscene. Process 800 may be implemented on control circuitry 402. In addition, one or more actions of process 800 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.

[0050] At 802, control circuitry 402 identifies multiple interactive paths within an interactive cutscene. For example, metadata describing an entire cutscene may include a counter or other data indicating the number of interactive paths in the cutscene. At 804, control circuitry 402 initializes a counter variable N and sets its value to 1, and a variable T represents the total number of interactive paths in the cutscene.

[0051] At 806, the control circuitry 402 determines whether the Nth interactive path has been selected. For example, if an interactive path has been selected, the control circuitry 402 may record a path ID or other unique identifier of the selected path. Alternatively, the control circuitry 402 may identify the selected path based on metadata of a frame of a cutscene currently being encoded for streaming. If the Nth interactive path has been selected ("Yes" at 806), then at 808, the control circuitry 402 may identify a supplemental content item associated with the Nth interactive path. For example, if the Nth interactive path advances the setting of a cutscene to an armory, the control circuitry 402 may identify a supplemental content item associated with an in-game offer for a special weapon.

[0052] If the Nth bidirectional path has not been selected ("No" at 806), then at 810 the control circuitry 402 determines whether N is equal to T, meaning all bidirectional paths have been checked. If N is not equal to T ("No" at 810), then at 812 the control circuitry 402 increments the value of N by 1 and processing returns to 806. If N is equal to T ("Yes" at 810), then all bidirectional paths have been checked and the process ends.

[0053] The actions or descriptions of Figure 8 may be used with any other embodiment of the present disclosure. In addition, the actions and descriptions described in connection with Figure 8 may be performed in alternative orders or in parallel as suitable to further the objectives of the present disclosure.

[0054] The process described above is intended to be illustrative, not limiting. Those skilled in the art will understand that any of the steps of the process discussed herein may be omitted, modified, combined, and / or rearranged, and any additional steps may be implemented without departing from the scope of the present invention. More generally, the above disclosure is meant to be illustrative, not limiting. Only the following claims are meant to set boundaries on what the invention includes. Furthermore, it should be noted that the features and limitations described in any one embodiment may be applied to any other embodiment herein, and that the flow charts or examples related to one embodiment may be combined with any other embodiment in a suitable manner, performed in a different order, or performed in parallel. In addition, the systems and methods described herein may be performed in real time. It should also be noted that the systems and / or methods described above may be applied to or used in accordance with other systems and / or methods. This specification discloses embodiments including, but not limited to, the following. 1. A method for inserting supplemental content into a cloud-rendered video game cutscene, the method comprising: Streaming video game cut scenes to a client device; and Retrieving game data associated with a game session associated with a client device; identifying supplemental content items for insertion into the video game cutscenes based on the game data; Detecting an insertion point within a video game cut scene; In response to detecting an insertion point within a video game cutscene, Streaming supplemental content to a client device in lieu of video game cutscenes; and and resuming streaming of the video game cut scenes to the client device upon receiving an indication that streaming of the supplemental content to the client device has ended. A method comprising: 2. The method further comprising inputting the raw multimedia data of the video game cut scene into a multimedia encoder for encoding in a streaming media format; Streaming supplemental content to a client device in lieu of the video game content is pausing input of raw multimedia data of the video game cut scene into a multimedia encoder; inputting raw multimedia data of the supplemental content into a multimedia encoder for encoding in a streaming format; Further comprising: Resuming streaming of video game cutscenes to client devices is ceasing input of raw multimedia data of the supplemental content to the multimedia encoder; Resuming input of raw multimedia data of video game cut scenes into a multimedia encoder; The method of claim 1, further comprising: 3. The method of claim 1, wherein streaming supplemental content to the client device in place of the video game cut scenes further includes pausing streaming of the video game cut scenes to the client device. 4. The method of claim 1, wherein the supplemental content is an advertisement. 5. Identifying supplemental content items for insertion into video game cutscenes based on the game data, reading a parameter from game data, the parameter having a value; comparing the value of the parameter to a reference value; selecting a supplemental content item based on the comparison; The method according to item 1, comprising: 6. The method of claim 5, wherein the parameters describe network conditions. 7. Comparing the value of a parameter with a reference value retrieving a metric value, the metric value describing a threshold level for a network condition described by the parameter; Determining whether the parameter value meets or exceeds a criterion value Further comprising: The method of claim 6, wherein selecting a supplemental content item based on the comparison further includes identifying a supplemental content item associated with a network condition described by the parameter in response to determining that the value of the parameter does not meet or exceed the reference value. 8. The method according to item 5, wherein the parameters describe in-game conditions. 9. Comparing the value of a parameter with a reference value retrieving a reference value, the reference value describing a first performance level; Determining whether the parameter value meets or exceeds a criterion value Further comprising: Selecting the supplemental content item based on the comparison includes: identifying a supplemental content item associated with a second competency level higher than the first competency level in response to determining that the value of the parameter meets or exceeds the criterion value; identifying a supplemental content item associated with the first proficiency level in response to determining that the value of the parameter does not meet or exceed the criterion value; The method of claim 8, further comprising: 10. The video game cutscene is an interactive cutscene, and identifying a supplemental content item for insertion into the video game cutscene based on the game data includes Identifying a plurality of interactivity paths within an interactive cutscene; determining whether a first bidirectional path of a plurality of bidirectional paths has been selected based on the game data; in response to determining that the first interactive path is selected, identifying a supplemental content item associated with the first interactive path; The method of claim 1, further comprising: 11. The method of item 1, wherein the video game cutscene is an interactive cutscene, and detecting an insertion point within the video game cutscene further comprises detecting an insertion point within a selected interactivity path of the interactive cutscene. 12. A system for inserting supplemental content into a cloud-rendered video game cutscene, comprising: Memory, Control circuit network and The control circuitry comprises: Streaming video game cut scenes to a client device; and Retrieving from the memory game data associated with the game session associated with the client device; identifying supplemental content items for insertion into the video game cutscenes based on the game data; Detecting an insertion point within a video game cut scene; In response to detecting an insertion point within a video game cutscene, Streaming supplemental content to a client device in lieu of video game cutscenes; and and upon receiving an indication that streaming of the supplemental content to the client device has ended, resuming streaming of the video game cut scenes to the client device. A system configured to: 13. The control circuitry is further configured to input raw multimedia data of the video game cut scenes to a multimedia encoder for encoding in a streaming media format; control circuitry configured to stream supplemental content to a client device in lieu of the video game content, pausing input of raw multimedia data of the video game cut scene into a multimedia encoder; inputting raw multimedia data of the supplemental content into a multimedia encoder for encoding in a streaming format;

[0023] 20. The method according to claim 1, further comprising: control circuitry configured to resume streaming of the video game cut scene to the client device, ceasing input of raw multimedia data of the supplemental content to the multimedia encoder; Resuming input of raw multimedia data of video game cut scenes into a multimedia encoder; Item 13. The system of item 12, further configured to: 14. The system of item 12, wherein the control circuitry configured to stream supplemental content to the client device in place of the video game cut scenes is further configured to pause streaming of the video game cut scenes to the client device. 15. The system of claim 12, wherein the supplemental content is an advertisement. 16. Control circuitry configured to identify supplemental content items for insertion into a video game cut scene based on game data, the control circuitry comprising: reading a parameter from game data, the parameter having a value; comparing the value of the parameter to a reference value; selecting a supplemental content item based on the comparison; Item 13. The system of item 12, further configured to: 17. The system of item 16, wherein the parameters describe network conditions. 18. Control circuitry configured to compare a value of a parameter to a reference value, comprising: retrieving a metric value, the metric value describing a threshold level for a network condition described by the parameter; Determining whether the parameter value meets or exceeds a criterion value

[0023] 20. The method according to claim 1, further comprising: Item 18. The system of item 17, wherein the control circuitry configured to select supplemental content items based on the comparison is further configured to, in response to determining that the value of the parameter does not meet or exceed the reference value, identify a supplemental content item associated with the network condition described by the parameter. 19. The system of item 16, wherein the parameters describe in-game conditions. 20. Control circuitry configured to compare a value of a parameter to a reference value, comprising: retrieving a reference value, the reference value describing a first performance level; Determining whether the parameter value meets or exceeds a criterion value

[0023] 20. The method according to claim 1, further comprising: control circuitry configured to select the supplemental content item based on the comparison, identifying a supplemental content item associated with a second competency level higher than the first competency level in response to determining that the value of the parameter meets or exceeds the criterion value; identifying a supplemental content item associated with the first proficiency level in response to determining that the value of the parameter does not meet or exceed the criterion value; 20. The system of item 19, further configured to: 21. The video game cut scene is an interactive cut scene, and the control circuitry configured to identify a supplemental content item for insertion into the video game cut scene based on the game data comprises: Identifying a plurality of interactivity paths within an interactive cutscene; determining whether a first bidirectional path of a plurality of bidirectional paths has been selected based on the game data; in response to determining that the first interactive path is selected, identifying a supplemental content item associated with the first interactive path; Item 13. The system of item 12, further configured to: 22. The system of item 12, wherein the video game cutscene is an interactive cutscene, and the control circuitry configured to detect an insertion point within the video game cutscene is further configured to detect an insertion point within a selected interactivity path of the interactive cutscene. 23. A system for inserting supplemental content into a cloud-rendered video game cutscene, the system comprising: means for streaming video game cut scenes to a client device; means for retrieving game data associated with a game session associated with a client device; means for identifying supplemental content items for insertion into video game cut scenes based on the game data; a means for detecting an insertion point within a video game cut scene; In response to detecting an insertion point within a video game cutscene, Streaming supplemental content to a client device in lieu of video game cut scenes; and and upon receiving an indication that streaming of the supplemental content to the client device has ended, resuming streaming of the video game cut scenes to the client device. and the means to The system comprises: 24. A method for providing a video game cut scene to a multimedia encoder for encoding in a streaming media format, comprising: The means for streaming supplemental content to a client device in lieu of the video game content includes: means for pausing input of raw multimedia data of the video game cut scene into a multimedia encoder; means for inputting raw multimedia data of the supplemental content to a multimedia encoder for encoding in a streaming format; Further equipped with The means to resume streaming of video game cut scenes to a client device is means for stopping the input of raw multimedia data of the supplemental content to the multimedia encoder; Means for resuming input of raw multimedia data of a video game cut scene into a multimedia encoder; 24. The system of claim 23, further comprising: 25. The system of item 23, wherein the means for streaming supplemental content to the client device in place of the video game cut scenes further comprises means for pausing streaming of the video game cut scenes to the client device. 26. The system of item 23, wherein the supplemental content is an advertisement. 27. A means for identifying a supplemental content item for insertion into a video game cut scene based on game data includes: means for reading a parameter from game data, the parameter having a value; means for comparing the value of the parameter with a reference value; means for selecting a supplemental content item based on the comparison; 24. The system according to item 23, comprising: 28. The system of item 27, wherein the parameters describe network conditions. 29. The means for comparing the value of a parameter with a reference value is means for retrieving a metric, the metric describing a threshold level for a network condition described by the parameter; A means of determining whether a parameter value meets or exceeds a criteria value; Further equipped with 29. The system of claim 28, wherein the means for selecting a supplemental content item based on the comparison further comprises means for identifying a supplemental content item associated with a network condition described by the parameter in response to determining that the value of the parameter does not meet or exceed the reference value. 30. The system of item 27, wherein the parameters describe in-game conditions. 31. The means for comparing the value of a parameter with a reference value is means for retrieving a reference value, the reference value describing a first performance level; A means of determining whether a parameter value meets or exceeds a criteria value; Further equipped with The means for selecting a supplemental content item based on the comparison comprises: means for identifying supplemental content items associated with a second competency level higher than the first competency level in response to determining that the value of the parameter meets or exceeds the criterion value; means for identifying a supplemental content item associated with a first proficiency level in response to determining that the value of the parameter does not meet or exceed the criterion value; Further comprising: Item 31. The system according to item 30. 32. The video game cutscene is an interactive cutscene, and the means for identifying a supplemental content item for insertion into the video game cutscene based on the game data includes: a means for identifying a plurality of interactivity paths within an interactive cutscene; means for determining whether a first bidirectional path of a plurality of bidirectional paths has been selected based on the game data; means for identifying a supplemental content item associated with the first interactive path in response to determining that the first interactive path has been selected; 24. The system of claim 23, further comprising: 33. The system described in item 23, wherein the video game cutscene is an interactive cutscene, and the means for detecting an insertion point within the video game cutscene further comprises means for detecting an insertion point within a selected interactive path of the interactive cutscene. 34. A non-transitory computer readable medium having encoded thereon non-transitory computer readable instructions for inserting supplemental content into a cloud-rendered video game cutscene, the instructions, when executed by control circuitry, performing: Streaming video game cut scenes to a client device; and Retrieving game data associated with a game session associated with a client device; identifying supplemental content items for insertion into the video game cutscenes based on the game data; Detecting an insertion point within a video game cut scene; In response to detecting an insertion point within a video game cutscene, Streaming supplemental content to a client device in lieu of video game cutscenes; and and upon receiving an indication that streaming of the supplemental content to the client device has ended, resuming streaming of the video game cut scenes to the client device. A non-transitory computer readable medium for causing control circuitry to perform the steps of: 35. Execution of the instructions further causes the control circuitry to input raw multimedia data of the video game cut scenes to a multimedia encoder for encoding in a streaming media format; Execution of the instructions to stream supplemental content to the client device in lieu of the video game content includes: pausing input of raw multimedia data of the video game cut scene into a multimedia encoder; inputting raw multimedia data of the supplemental content into a multimedia encoder for encoding in a streaming format; and further causing the control circuitry to Execution of the instruction to resume streaming of the video game cutscene to the client device ceasing input of raw multimedia data of the supplemental content to the multimedia encoder; Resuming input of raw multimedia data of video game cut scenes into a multimedia encoder; Item 35. The non-transitory computer readable medium of item 34, further causing control circuitry to: 36. The non-transitory computer-readable medium of item 34, wherein execution of the instructions for streaming supplemental content to the client device in place of the video game cut scenes further causes the control circuitry to pause streaming of the video game cut scenes to the client device. 37. The non-transitory computer-readable medium of item 34, wherein the supplemental content is an advertisement. 38. Execution of the instructions for identifying supplemental content items for insertion into a video game cut scene based on game data includes: reading a parameter from game data, the parameter having a value; comparing the value of the parameter to a reference value; selecting a supplemental content item based on the comparison; Item 35. The non-transitory computer readable medium of item 34, further causing control circuitry to: 39. The non-transitory computer-readable medium of item 38, wherein the parameters describe network conditions. 40. Execution of an instruction to compare the value of a parameter with a reference value retrieving a metric value, the metric value describing a threshold level for a network condition described by the parameter; Determining whether the parameter value meets or exceeds a criterion value and further causing the control circuitry to 40. The non-transitory computer-readable medium of claim 39, wherein execution of the instructions for selecting a supplemental content item based on the comparison further causes the control circuitry to, in response to determining that the value of the parameter does not meet or exceed the reference value, identify a supplemental content item associated with the network condition described by the parameter. 41. The non-transitory computer-readable medium of item 38, wherein the parameters describe in-game conditions. 42. Execution of an instruction to compare the value of a parameter with a reference value retrieving a reference value, the reference value describing a first performance level; Determining whether the parameter value meets or exceeds a criterion value and further causing the control circuitry to Execution of the instructions for selecting the supplemental content item based on the comparison includes: identifying supplemental content items associated with a second competency level higher than the first competency level in response to determining that the value of the parameter meets or exceeds the criterion value; identifying a supplemental content item associated with the first proficiency level in response to determining that the value of the parameter does not meet or exceed the criterion value; Item 42. The non-transitory computer-readable medium of item 41, further causing the control circuitry to: 43. The video game cutscene is an interactive cutscene, and execution of the instructions to identify a supplemental content item for insertion into the video game cutscene based on the game data includes: Identifying a plurality of interactivity paths within an interactive cutscene; determining whether a first bidirectional path of a plurality of bidirectional paths has been selected based on the game data; in response to determining that the first interactive path is selected, identifying a supplemental content item associated with the first interactive path; Item 35. The non-transitory computer readable medium of item 34, further causing control circuitry to: 44. The non-transitory computer-readable medium of item 34, wherein the video game cut scene is an interactive cut scene, and execution of the instructions for detecting an insertion point within the video game cut scene further causes the control circuitry to detect an insertion point within a selected interactive path of the interactive cut scene. 45. A method of inserting supplemental content into a cloud-rendered video game cutscene, the method comprising: receiving multimedia data of a video game cut scene; Streaming video game cut scenes to a client device; and Retrieving game data associated with a game session associated with a client device; identifying supplemental content items for insertion into the video game cutscenes based on the game data; Detecting an insertion point within a video game cut scene; In response to detecting an insertion point within a video game cutscene, Streaming supplemental content to a client device in lieu of video game cutscenes; and and upon receiving an indication that streaming of the supplemental content to the client device has ended, resuming streaming of the video game cut scenes to the client device. A method comprising: 46. ​​The method further comprising inputting raw multimedia data of the video game cut scenes to a multimedia encoder for encoding in a streaming media format; Streaming supplemental content to a client device in lieu of the video game content is pausing input of raw multimedia data of the video game cut scene into a multimedia encoder; inputting raw multimedia data of the supplemental content into a multimedia encoder for encoding in a streaming format; Further comprising: Resuming streaming of video game cutscenes to client devices is ceasing input of raw multimedia data of the supplemental content to the multimedia encoder; Resuming input of raw multimedia data of video game cut scenes into a multimedia encoder; Item 46. The method of item 45, further comprising: 47. The method of any of items 45-46, wherein streaming supplemental content to the client device in place of the video game cut scenes further includes pausing streaming of the video game cut scenes to the client device. 48. The method of any one of items 45-47, wherein the supplemental content is an advertisement. 49. Identifying supplemental content items for insertion into video game cut scenes based on game data includes: reading a parameter from game data, the parameter having a value; comparing the value of the parameter to a reference value; selecting a supplemental content item based on the comparison; 49. The method according to any one of items 45 to 48, comprising: 50. The method of claim 49, wherein the parameters describe network conditions. 51. Comparing the value of a parameter to a reference value retrieving a metric value, the metric value describing a threshold level for a network condition described by the parameter; Determining whether the parameter value meets or exceeds a criterion value Further comprising: The method of claim 50, wherein selecting a supplemental content item based on the comparison further includes, in response to determining that the value of the parameter does not meet or exceed the reference value, identifying a supplemental content item associated with the network condition described by the parameter. 52. The method according to item 49, wherein the parameters describe in-game conditions. 53. Comparing the value of the parameter to a reference value further comprises: retrieving a reference value, the reference value describing a first performance level; determining whether the value of the parameter meets or exceeds a reference value; Including, Selecting the supplemental content item based on the comparison includes: identifying a supplemental content item associated with a second competency level higher than the first competency level in response to determining that the value of the parameter meets or exceeds the criterion value; identifying a supplemental content item associated with the first proficiency level in response to determining that the value of the parameter does not meet or exceed the criterion value; 53. The method of claim 52, further comprising: 54. The video game cutscene is an interactive cutscene, and identifying a supplemental content item for insertion into the video game cutscene based on the game data includes: Identifying a plurality of interactivity paths within an interactive cutscene; determining whether a first bidirectional path of a plurality of bidirectional paths has been selected based on the game data; in response to determining that the first interactive path is selected, identifying a supplemental content item associated with the first interactive path; 54. The method according to any one of items 45-53, further comprising: 55. The method of any of items 45-54, wherein the video game cutscene is an interactive cutscene, and detecting an insertion point within the video game cutscene further includes detecting an insertion point within a selected interactivity path of the interactive cutscene.

Claims

1. A method for inserting supplementary content into a cloud-rendered video game cutscene, wherein the method is: Streaming video game cutscenes to client devices, This involves reading game data related to the game session associated with the aforementioned client device, Based on the aforementioned game data, identify supplementary content items for insertion into the video game cutscene, To detect the insertion point within the aforementioned video game cutscene, In response to detecting the insertion point within the video game cutscene, In place of the aforementioned video game cutscene, the supplemental content is streamed to the client device. Upon receiving an instruction that the streaming of the supplemental content to the client device has ended, the system will resume streaming the video game cutscene to the client device. Methods that include...

2. Further comprising inputting the raw multimedia data of the video game cutscene into a multimedia encoder for encoding in a streaming media format, Streaming the supplemental content to the client device instead of the video game content is: Pausing the input of the raw multimedia data of the video game cutscene to the multimedia encoder, Inputting the raw multimedia data of the supplemental content into the multimedia encoder for encoding in the aforementioned streaming format. It further includes, Resuming the streaming of the video game cutscene to the client device is: Stopping the input of the raw multimedia data of the supplemental content to the multimedia encoder, To resume input of the raw multimedia data of the video game cutscene to the multimedia encoder. The method according to claim 1, further comprising:

3. The method according to any one of claims 1-2, wherein streaming the supplemental content to the client device instead of the video game cutscene further includes pausing the streaming of the video game cutscene to the client device.

4. The method according to claim 1, wherein the supplementary content is an advertisement.

5. Based on the aforementioned game data, identifying supplemental content items for insertion into the video game cutscene is: The process involves reading parameters from the aforementioned game data, wherein the parameters have values. The aforementioned value of the parameter is compared with the reference value, Based on the above comparison, select supplemental content items. The method according to claim 1, including the method described in claim 1.

6. The method according to claim 5, wherein the parameters describe network conditions.

7. Comparing the aforementioned values ​​of the parameters with reference values ​​means Reading the aforementioned reference value, wherein the reference value describes a threshold level for the network condition described by the parameter, To determine whether the value of the parameter satisfies or exceeds the reference value. It further includes, The method of claim 6, wherein selecting a supplemental content item based on the comparison further includes identifying a supplemental content item related to the network condition described by the parameter in response to determining whether the value of the parameter neither satisfies nor exceeds the threshold value.

8. The method according to claim 5, wherein the parameters describe in-game conditions.

9. Comparing the aforementioned values ​​of the parameters with reference values ​​means The process involves reading out the aforementioned reference value, wherein the aforementioned reference value describes a first competency level. To determine whether the value of the parameter satisfies or exceeds the reference value. It further includes, Based on the above comparison, selecting supplemental content items is: In response to determining whether the value of the parameter satisfies or exceeds the threshold value, identify supplemental content items associated with a second capability level higher than the first capability level, In response to determining that the value of the parameter neither meets nor exceeds the threshold value, identify supplemental content items related to the first capability level. The method according to claim 8, further comprising:

10. The aforementioned video game cutscene is an interactive cutscene, and based on the game data, it is possible to identify supplemental content items for insertion into the video game cutscene. Identifying multiple bidirectional paths within the aforementioned bidirectional cutscene, Based on the aforementioned game data, it is determined whether the first bidirectional path of the plurality of bidirectional paths has been selected. In response to determining that the first bidirectional path has been selected, identify the supplemental content items associated with the first bidirectional path. The method according to claim 1, further comprising:

11. The method according to claim 1, wherein the video game cutscene is a bidirectional cutscene, and detecting an insertion point in the video game cutscene further comprises detecting an insertion point in a selected bidirectional path of the bidirectional cutscene.

12. A system configured to carry out the method described in claim 1.

13. A system for inserting supplementary content into a cloud-rendered video game cutscene, wherein the system is Memory and Control network and Equipped with, The aforementioned control circuit network is Streaming video game cutscenes to client devices, Reading game data related to the game session associated with the client device from the memory, Based on the aforementioned game data, identify supplementary content items for insertion into the video game cutscene, To detect the insertion point within the aforementioned video game cutscene, In response to detecting the insertion point within the video game cutscene, Streaming the aforementioned supplemental content to the client device, Upon receiving an instruction that the streaming of the supplemental content to the client device has ended, the system will resume streaming the video game cutscene to the client device. A system configured to perform the following actions.

14. The control circuit network is further configured to input raw data of the video game cutscene to an encoder for encoding in a streaming media format. The control circuit network configured to stream the supplemental content to the client device is: The input of the raw data of the video game cutscene to the encoder is temporarily suspended, Inputting the raw data of the supplemental content into the encoder for encoding in the aforementioned streaming format It is further configured to do the following: The control network configured to resume streaming the video game cutscene to the client device, Stopping the input of the raw data of the supplemental content to the encoder, To resume inputting the raw data of the video game cutscene to the encoder. The system according to claim 13, further configured to perform the following:

15. The aforementioned video game cutscene is a bidirectional cutscene, and the control circuit network is configured to identify supplemental content items for insertion into the video game cutscene based on the game data, Identifying multiple bidirectional paths within the aforementioned bidirectional cutscene, Based on the aforementioned game data, it is determined whether the first bidirectional path of the plurality of bidirectional paths has been selected. In response to determining that the first bidirectional path has been selected, identify the supplemental content items associated with the first bidirectional path. The system according to claim 13, further configured to perform the following:

Images