Camera view selection processor for passive audience viewing

Abstract

The invention relates to a camera view selection processor for passive viewer viewing. A method includes receiving, from a passive viewer, a request to view a video game executed at a game cloud server, the video game being a multi-player video game that generates game play data using inputs from a plurality of players, the game proceeding data is used to generate a player camera view for each of the plurality of players and an active viewer camera view for each active viewer proceeding for the game viewing the video game; analyzing the user profile of the passive viewer to determine viewing preferences of the passive viewer; and identifying a selected camera view of the player camera view and the active viewer camera view for forwarding to a client device associated with the passive viewer, the camera views are used for rendering specific camera views of the player camera views and the active viewer camera views identified based on the passive viewer user profiles and corresponding to actions currently occurring in the video game during game progress.

Description

[0001] This application is a divisional application of U.S. nonprovisional patent application No. 16 / 886,727, filed May 28, 2020, application number 202180060860.2, filed May 5, 2021, entitled “Camera View Selection Processor for Passive Viewer Viewing”. Technical Field

[0002] This disclosure relates to providing viewers with a recommended stream of gameplay footage of a video game, and more specifically to providing a recommended stream of different camera views of a game scene that capture different aspects of gameplay. Background Technology

[0003] In recent years, video games have become increasingly popular. There are two main types of video games. In one type, the video game runs on a client device (e.g., a mobile device, PC, laptop, etc.) and the client connects to a server to share game-related metadata. The other type is streaming video games, where the game runs on one or more servers as part of a game cloud and game data is streamed to client devices for rendering. Streaming video games, especially massively multiplayer online (MMO) games, are becoming increasingly popular because a large number of users (e.g., players, viewers) can simultaneously access the streaming video game via computer networks distributed across a wide geographical area.

[0004] The development trend of video games is the concept of cloud gaming. Cloud gaming enables video games to run on one or more cloud servers using resources available in a game cloud system and can be accessed by multiple players and viewers from anywhere with a network connection. Massive multiplayer online games (MMOs) are an example of video games utilizing cloud gaming, and these games are becoming increasingly popular. With the current trend of including complex graphics in video games (e.g., MMOs), video games require a large amount of computing resources, and cloud gaming provides the necessary computing resources in a centralized location to meet the needs of modern game engines. Video games run on one or more servers in a game cloud system and the game data is streamed to the user's client device. The advantage of running video games on a game cloud system is the centralization of the resources required to run the game, thereby eliminating the need for the user's client device to have specific hardware to run the video game. The client device only needs to have hardware sufficient to receive, decode, and render the game while enjoying a high-quality video game experience.

[0005] One or more players play a video game, and multiple spectators can tune in to watch the game play of the one or more players. Player inputs provided during game play are used to update a game state of the video game and generate game play data that is streamed to client devices of the one or more players and spectators. Game scenes of the video game are rendered on the respective client devices of the players and spectators using the game play data. As cloud gaming is becoming more popular, the graphics complexity configured for cloud gaming and thus the quality of the video game also continues to increase.

[0006] It is in this context that embodiments of the present disclosure are presented. SUMMARY

[0007] Embodiments of the present disclosure relate to systems and methods for recommending a stream of game play of a video game to spectators interested in the game play of the video game. The video game can be a single-user video game or a massively multiplayer online (MMO) video game that is executable on a cloud gaming system and is accessed by multiple players at the same location or remotely located (i.e., distributed at different geographical locations) for game play. Game inputs provided by the players are used to affect outcomes of the game and generate game play data. The game play data is processed to generate frames of game scenes that are streamed to the client devices of the players for rendering. The frames of game scenes provided to each of the players provide a camera view of the game scenes of the video game from the perspective of the respective player (i.e., a “player camera view”). Multiple spectators can be interested in watching the game play of the video game. Some of the multiple spectators can choose to actively participate in watching the game play of the video game by selecting a particular camera view to watch the game play, while other spectators can simply choose the video game to watch. For the spectators that actively participate in selecting a particular camera view (referred to herein as “active spectators”), the system can generate the particular camera view (i.e., an active spectator camera view) and forward the particular camera view to the spectators that requested the particular camera view. For the spectators that simply choose the video game to watch without actively specifying any particular camera view, the system determines which of the camera views (i.e., the player camera view or the active spectator camera view) to provide to these spectators. These spectators that simply choose to watch the video game without actively selecting any particular camera view are also referred to herein as “passive spectators”. For the passive spectators, the system switches between the different camera views generated for the video game such that at any given time, the passive spectators are presented with either the player camera view of a player or the active spectator camera view of an active spectator. The switching between the two camera views is based on the context of the action occurring in the respective camera view. During the course of the video game, the passive spectators are presented with multiple camera views (player camera views or active spectator camera views), each of which presents a view of the action occurring in the video game. The multiple camera views provided to the passive spectators constitute a recommended stream. The camera views included in the recommended stream are a subset of the camera views captured during the game play and provide interesting details of the video game to the passive spectators.

[0008] In some aspects, a method includes receiving a request from a passive spectator to watch a video game executing at a game cloud server, wherein the video game is a multi-player video game that generates game play data using inputs from a plurality of players, the game play data used to generate a player camera view for each of the plurality of players and an active spectator camera view for each active spectator watching a game play of the video game; analyzing a user profile of the passive spectator to determine a viewing preference of the passive spectator; and identifying a curated camera view of the player camera views and the active spectator camera views for forwarding to a client device associated with the passive spectator for rendering a particular camera view of the player camera views and the active spectator camera views identified based on the user profile of the passive spectator and corresponding to an action currently occurring in the video game during game play.

[0009] In one embodiment, a method is provided. The method includes executing a video game at a game cloud server in response to a request received from a player to play a game of the video game. The execution of the video game causes generation of game play data, processing of the game play data to generate a player camera view during game play to provide to the player, and generation of an active spectator camera view to provide to an active spectator accessing the video game to spectate the game play of the player. A request is received from a passive spectator to watch the game play of the video game. In response to the request, one of the player camera view or the active spectator camera view is presented to the passive spectator. The presenting includes dynamically switching between the player camera view and the active spectator camera view. The dynamic switching is based on a context of an action occurring in the player camera view and the active spectator camera view. The dynamic switching is carried out without any input from the passive spectator.

[0010] In one embodiment, the player camera view and the active spectator camera view are dynamically updated during game play to capture a view of the action occurring in the video game.

[0011] In one embodiment, the player camera view or the active spectator camera view selected for the dynamic switching is related in time to the action occurring in the video game.

[0012] In one embodiment, the dynamic switching to the player camera view or the active spectator camera view is further based on a profile of the passive spectator.

[0013] In one embodiment, the profile of the passive spectator identifies a viewing preference of the passive spectator collected over time based on content viewed by the passive spectator.

[0014] In one implementation, the passive viewer's viewing preferences are dynamically adjusted in the configuration file based on preference input received from the passive viewer.

[0015] In one implementation, the actions occurring in the video game are based on activities performed by the player, and the player's camera view and the active viewer's camera view capture the actions in relation to those activities.

[0016] In one implementation, the dynamic switching continues during gameplay of the video game and generates a recommended stream for the passive audience. The recommended stream includes a combination of one or more player camera views and / or one or more active audience camera views. Each player camera view and each active audience camera view is dynamically generated and includes dissimilar actions occurring within the video game.

[0017] In one implementation, multiple actions occur during gameplay of the video game, each of which is associated with one or more player camera views and one or more active viewer camera views. The one or more player camera views and the one or more active viewer camera views associated with each action capture disparate views of the game scene in the video game where the corresponding action occurs. The one or more player camera views and the one or more active viewer camera views generated for the action include sufficient data to construct a three-dimensional representation of the game scene of the video game associated with the action.

[0018] In one implementation, processing the game progress data includes re-enforcing the video game to generate an active viewer camera view that captures the desired angles of the actions occurring in the video game, specified by input from the active viewer. This re-enforcing is performed using game progress data available when input from the active viewer is received.

[0019] In one implementation, the player camera view captures a view of the actions occurring in the video game from the player's perspective. The active viewer camera view is generated based on input provided by the active viewer. The active viewer camera view captures a view of actions different from those captured in the player camera view.

[0020] In one implementation, the dynamic switching includes presenting the player's camera view or the active viewer's camera view to the passive viewer during the duration of an action occurring in the video game.

[0021] In one embodiment, the game play data is an aggregation of game play data of multiple players currently playing the video game. The aggregated game play data is used to generate a player camera view for each of the multiple players and an active spectator camera view for the active spectator for each action occurring in the video game. Each player camera view captures a view of the video game from a perspective of one of the multiple players using the player's game input, and each active spectator camera view captures a view of the video game according to a view specification provided in the input of a particular active spectator. The dynamic switching includes presenting only a player camera view, or only an active spectator camera view, or a combination of a player camera view and an active spectator camera view, thereby capturing a view of a game scene associated with different actions occurring in the video game.

[0022] In one embodiment, the dynamic switching is further based on a profile of the passive spectator.

[0023] In one embodiment, the dynamic switching is implemented by generating a recommendation model using machine learning logic. The recommendation model is generated and trained using game play data of the video game. The game play data is generated in response to the game input of the players, the input of the active spectator. An output related to a context of an action occurring in the video game is identified from the recommendation model. The output is used to identify the player camera view or the active spectator camera view for dynamic switching.

[0024] In another embodiment, a method is disclosed. The method includes detecting a request to watch a video game executing at a game cloud server. The request is received from a passive spectator. A plurality of camera views generated during game play of the video game are identified. The plurality of camera views include player camera views generated for each player and active spectator camera views generated for each active spectator focusing on the game play of the video game, wherein each camera view of the plurality of camera views captures a distinct view of a game scene of the video game associated with a corresponding action occurring in the video game. The method further includes dynamically switching between a player camera view and an active spectator camera view selected from the plurality of camera views associated with each action in response to the request to return to the passive spectator. The dynamic switching occurs without any input from the passive spectator and is based on a context of the action emerging in the respective player camera view and the active spectator camera view selected for the action.

[0025] Other aspects and advantages of the present disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0026] The present disclosure can best be understood with reference to the following description and drawings.

[0027] Figure 1 FIG. illustrates a simplified conceptual game cloud system for collecting game inputs from players at different geographical locations and adjusting video game outputs for providing a recommended stream to viewers, in accordance with one embodiment of the present disclosure.

[0028] Figure 2A FIG. illustrates a simplified diagram of different camera views for game play capture of a video game for providing a recommended stream, in accordance with one embodiment of the present disclosure. Figure 2B FIG. illustrates an exploded view illustrating different camera views captured for different users (players, active viewers), in accordance with one embodiment of the present disclosure.

[0029] Figure 3A FIG. illustrates a simplified block diagram of a game cloud system for collecting and processing game inputs from multiple players and inputs from multiple active viewers to generate corresponding player camera views and active viewer camera views for each of the multiple players and active viewers, in accordance with one embodiment of the present disclosure.

[0030] Figure 3B FIG. illustrates a simplified block diagram of a game cloud system for providing different camera views to passive viewers, in accordance with one embodiment of the present disclosure.

[0031] Figure 4 FIG. illustrates a simplified conceptual block diagram of a view recommendation engine for generating different camera views that constitute a recommended stream provided to passive viewers, in accordance with one embodiment of the present disclosure.

[0032] Figure 5 FIG. illustrates a simplified block diagram of different modules within a view recommendation engine for generating a recommended stream for viewers, in accordance with one embodiment of the present disclosure.

[0033] Figure 6 FIG. illustrates a simplified block diagram of different modules within a feature processing engine for extracting and evaluating various features of game play data provided to a recommendation engine for identifying different camera views to present to passive viewers, in accordance with one embodiment of the present disclosure.

[0034] Figures 7A-1 to 7E-1 FIG. illustrates a sample of switched camera views presented to different passive viewers to watch game play of a video game, in accordance with one embodiment of the present disclosure. Figures 7A-2 to 7E-2 FIG. illustrates a sample of switched camera views presented to different passive viewers to watch game play of a video game, in accordance with one embodiment of the present disclosure. Figures 7A-1 to 7E-1The sample recommendation stream generated by the mid-identified switch camera view is provided by the view recommendation engine to different passive viewers.

[0035] Figure 8 A flow diagram illustrating various operations of a method for generating a recommendation stream for passing back to a viewer in accordance with one implementation of the present disclosure.

[0036] Figure 9 An exemplary implementation of an information service provider architecture in accordance with one implementation of the present disclosure is illustrated.

[0037] Figure 10 A simplified block diagram of an exemplary cloud game server for executing an instance of a video game in accordance with one implementation of the present disclosure is illustrated. DETAILED DESCRIPTION

[0038] In the following description, numerous specific details are set forth to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that the present disclosure can be practiced without some or all of these specific details. In other instances, well known process steps have not been described in detail in order not to obscure the present disclosure.

[0039] Currently, players select a video game (referred to herein as "game") to play and provide game inputs. The game logic processes the game inputs to generate game play data. The game play data is used to generate various camera views for each scene of the video game that a player accesses. The streaming capability of the video game enables live streaming of the game play via a network, such as the Internet. Spectators can access the video game and watch the game play of the players through a website, a game platform, a mobile application, a content provider platform, or any other application / content platform. The spectators can wish to focus on the game play of a particular player or the entire game play of the game. The complex design of the video game allows the video game to generate different camera views for each action that occurs in each game scene that a player accesses. For example, the game play data of the video game can be used to generate a camera view for each player playing the video game (i.e., a player camera view). The camera view generated for each player can capture a view of the game scene in which a particular action is occurring from the perspective of the player. In one implementation, the particular action can occur in response to the game inputs of the player. In an alternative implementation, the particular action can occur in response to the game inputs of other players or in response to the natural progression of the video game. In addition to the player camera view, the video game can also generate a spectator camera view for each spectator accessing the video game to watch the game play when the spectator wishes to focus on the game play of the game. The spectator can provide inputs to specify the camera view that he or she desires to watch for each action that occurs in each game scene of the video game, and the spectator camera view is generated according to the details provided by the spectator in the inputs. The view of the game scene captured in each spectator camera view generated for each action of the video game is different from the view captured in the player camera view generated for the same action. A spectator who actively provides inputs to specify the camera view that he or she desires to watch during the game play is referred to herein as an "active spectator." Thus, the spectator camera view generated for an active spectator is referred to herein as an "active spectator camera view" (ASCV). Alternatively, a spectator can wish to focus on the entire game play of the game. In this case, the spectator does not actively provide inputs to specify the camera view that he or she wants to see. Instead, the spectator relies on the system to provide the camera view for different actions that occur in the video game. Since the spectator does not actively provide inputs to select the camera view, the spectator is referred to herein as a "passive spectator." The player camera view or the active spectator camera view for each action is selected using the game logic to be presented to the passive spectator. According to the large number of players playing the MMO video game and the number of spectators watching the video game, the video game generates a large number of camera views that capture the view of different actions that occur in the video game during the game play.

[0040] Currently, when a passive spectator desires to watch a game play of a video game, the passive spectator is provided with all camera views generated for each action occurring in the video game. The passive spectator must manually navigate through each camera view of each action to select the particular camera view of each action that the passive spectator would like to watch. Given the large number of camera views generated for each action, this can be a daunting task. Having to manually navigate through various camera views is both time consuming and laborious, which can lead to the passive spectator becoming frustrated and losing interest in the game play of the game. A loss of interest by the spectator can lead to a loss of revenue for the game developer, as interest in the game can often translate into a purchase of the game by the spectator or higher revenue from a sponsor.

[0041] To address the problem, various embodiments of the present disclosure provide systems and methods that collect game play data of a game from a plurality of players from a game play session (e.g., a current session or a previous session), identify various camera views that can be taken from each action in a game scene, and intelligently identify particular camera views that will provide a passive spectator with a desired view of the action within the game scene. The particular camera view of each action is identified based on a profile of the passive spectator. The profile identifies viewing preferences of the passive spectator based on a viewing history of the passive spectator collected over time. In some instances, the viewing preferences of the passive spectator can be dynamically adjusted in accordance with preference inputs provided by the passive spectator. A recommended stream is generated using various camera views of the video game provided to the passive spectator during the viewing of the video game. The camera views presented to the passive spectator and included in the recommended stream can include only player camera views selected from the game play of a single player or from a plurality of players, only active spectator camera views generated for different active spectators, or a combination of both player camera views and active spectator camera views. As noted, the player camera views of a player encompass a view of the game scene from the perspective of the player in which the action occurs, and the active spectator camera views capture a game scene other than or different from the game scene captured in the player camera views. Each of the camera views included in the recommended stream is tailored to the profile of the passive spectator and selected to match one or more viewing preferences of the passive spectator. Selecting the camera views in accordance with the profile of the passive spectator enables the passive spectator to watch the game play in accordance with what the passive spectator would like to watch, and the selection of the camera views is made without the need for input from the passive spectator, thereby enabling the passive spectator to have an enriched viewing experience.

[0042] A view recommendation engine involving a machine learning algorithm is used to generate a recommendation model that utilizes game input from a player and game play data generated during game play of a video game. The recommendation model is continued to be trained with additional game input received from the player and additional game play data generated for the video game in response to the additional game input. An output is identified from the recommendation model to match a viewing target defined by a viewing preference included in a profile of a passive spectator. The output from the recommendation model is used to identify a particular camera view to present to the passive spectator, the particular camera view including a view of a game scene of the game where action is occurring. The particular camera view identified from the output of the recommendation model can be more relevant to the passive spectator than a view provided by the video game because the particular camera view is selected in accordance with the viewing preference of the passive spectator.

[0043] Having generally described an embodiment of the present application, exemplary details of various implementations will now be described with reference to the various figures.

[0044] Figure 1 An overview of a game cloud system (GCS) 10 for accessing games for play is provided. The GCS 10 includes a plurality of client devices 100 (100-1, 100-2, 100-3,... 100-n) distributed at a single geographic location or at different geographic locations and communicatively connected to a game cloud site 300 via a network 200. The GCS 10 is configured to host a plurality of games and other interactive applications, such as social media applications, content provider applications, and the like. The GCS 10 can be accessed from a single geographic location or from multiple geographic locations. The client devices 100 can be any type of client computing device having a processor, memory, and communication capabilities to access the network 200, such as a LAN, wired, wireless, or 4G / 5G, and the like, and can be portable or non-portable. The client devices 100 can run an operating system and include a network interface to access the network 200, or can be a thin client having a network interface to communicate with the game cloud site 300 via the network 200, where the game cloud site 300 provides computing functionality. By way of example, the client devices can be a smart phone, mobile device, tablet computer, desktop computer, personal computer, wearable device, connected television, or hybrid or other digital device including a monitor or touch screen having a portable form factor.

[0045] A client device 100 having 5G communication capabilities can include a mobile device or any other computing device capable of connecting to a 5G network. In one implementation, the 5G network is a digital cellular network in which service areas are divided into multiple "cells" (i.e., small geographic areas). Analog data generated at the mobile device is digitized and transmitted to a local antenna within the cell using radio waves in a frequency channel that can be used again in geographically separated cells. The local antenna is connected to the Internet and telephone networks through a high-bandwidth fiber-optic or other similar wireless communication. Because the 5G network uses higher radio frequency waves for communication, the 5G network is able to transmit data at higher data rates and thus provides lower network latency.

[0046] A player can access video games available at the GCS 10 using a user account. In response to a request from a player for access to a game to play the game, the player's user account is authenticated in accordance with the user accounts 304 maintained in the user data store 305. The request is also authenticated in accordance with the game data store 306 to determine whether the player is eligible to access and play the video game before providing access to the video game. The authentication is performed by identifying all game titles that the player is eligible to view or play available at the game cloud site 300 and confirming the game title included in the player's request in accordance with the identified game titles. The game data store 306 maintains a list of game titles that are at or can be hosted at the GCS 10, and as new games are introduced, the game data store 306 is updated with game titles, game codes, and information related to the new games. It should be noted that although various implementations are described with respect to video games (also referred to as "games"), the implementations can be extended to include any other interactive applications.

[0047] After successfully authenticating the user and the request, the game cloud site 300 identifies a data center that can host the game and sends a signal to the identified data center to load the game associated with the game name identified in the request. In some embodiments, more than one data center can host or be capable of hosting the game. In these embodiments, the game cloud site 300 identifies a data center that is geographically close to the player's geographic location. The player's geographic location can be determined using global positioning system (GPS) mechanisms within the client device 100, the client device's IP address, the client device's ping information, the player's social and other online interactions made via the client device 100, just to name a few. Of course, the foregoing ways of detecting the player's geographic location are provided as examples, and it should be noted that other types of mechanisms or tools can be used to determine the player's geographic location. Identifying a data center that is close to the player's geographic location can reduce latency when transmitting game-related data between the player's client device 100 and the game executing at the identified data center 301. The data center 301 can include a plurality of game servers 302, and the game server 302 is selected based on resources available at the game server 302 for hosting the game. In some embodiments, an instance of the game can be executed on one or more game servers 302 within the identified data center 301.

[0048] In some embodiments, the identified data center 301 can not have the resources (e.g., bandwidth, processing, etc.) required to host the game. In these embodiments, the game cloud site 300 can identify a second data center that is geographically close to the player's geographic location and has the resources required to host the game.

[0049] The game cloud site 300 loads the game into one or more game servers 302 in the identified data center 301. The one or more game servers 302 include hardware / software resources that satisfy the game's requirements. The game server 302 can be any type of server computing device available in the game cloud site 300, including a standalone server, etc. Further, the game server 302 can manage one or more virtual machines that support a game processor that executes an instance of the game for the player on a host.

[0050] In some embodiments, the one or more servers 302 can include a plurality of game consoles (or computing devices) 303, and the game cloud site 300 can identify one or more game consoles or computing devices 303 within the identified one or more servers 302 to load the game. Each of the one or more game consoles / computing devices 303 can be a standalone game console or computing device, or can be a rack-mounted server or a blade server. The blade server in turn can include a plurality of server blades, each of which has the circuitry and resources needed to instantiate a single instance of the game, for example. Of course, the game consoles described above are exemplary and should not be considered limiting. Other types of game consoles or computing devices, including other forms of blade servers, can also be used to execute an identified instance of a game. Once one or more game consoles or computing devices are identified, the generic game-related code of the game is loaded onto the one or more game consoles / computing devices and made available for access by the player.

[0051] In other embodiments, the video game can be executed locally at the client device 100, and the metadata resulting from executing the video game can be transmitted via the network 200 to a game cloud server (herein simply referred to as "game server") 302 at the identified data center 301 of the game cloud site 300 for use in influencing the game state and sharing game play data with other players and spectators.

[0052] Game inputs that affect the game state of the game can be provided from input devices such as a mouse 112, a keyboard (not shown), and the like, or a control interface associated with the client device 100, e.g., a touch screen, and the like, or from a controller 120 that is communicatively connected to the client device 100. A recommendation model (i.e., an artificial intelligence (AI) model) is formed using game play data generated from game inputs of the players during game play of the video game. The recommendation model is further trained using additional game play data generated with ongoing game inputs provided by the players during a game session. The trained recommendation model is used to identify camera views provided to each of the players. The camera views provided to each of the players capture views of game scenes in which one or more actions occur during game play of the video game, and these camera views are designed to capture the game scenes from the perspective of each of the players. Thus, the camera views provided to each of the players are also referred to herein as “player camera views.” The trained recommendation model is also used to identify camera views provided to different active spectators to satisfy viewing goals of each of the spectators. An active spectator is one who has accessed the video game to watch game play and actively provides inputs to identify different views of game scenes associated with each action of the video game that the active spectator wants to watch. Based on the inputs from the active spectators, the video game is executed again to generate specified camera views for the active spectators. The camera views generated for the active spectators are thus also referred to as “active spectator camera views” and capture views of game scenes that capture the actions from different viewing angles or viewing depths or viewing directions than captured in the camera views provided to each of the players.

[0053] In some instances, in addition to active viewers, viewers can access the video game to watch. In such a case, the viewers choose the video game to watch, but do not provide any other input to select a particular camera view. After selecting the video game, the viewers sit down to watch the camera views that capture different stages of gameplay of the video game. In such a case, the viewers are referred to hereinafter as "passive viewers" because they do not provide any input to affect the generation of the camera views, but rather rely on the system to provide them with camera views that capture different actions in the video game. The particular camera views are identified and provided to the passive viewers using a view recommendation engine that is executed at the game cloud site 300. The view recommendation engine uses a recommendation model to identify particular camera views in the camera views of the video game to return to the passive viewers. The camera views that capture different actions in the video game are identified according to the profile of the passive viewers. Each camera view identified for the passive viewers captures a view of a game scene where an action occurs and is presented for the duration of the action, then dynamically switches to another camera view that captures the next action that occurs in the video game. Thus, during the course of watching the gameplay of the video game, one or more player camera views and / or one or more active viewer camera views can be presented to the passive viewers.

[0054] The game executed at the game cloud site 300 can be a single-player game or a multi-player game. In some implementations, the video game can be a massively multiplayer online (MMO) game that allows multiple players from different geographical locations to access and play the video game. Thus, the player camera views presented to the passive viewers can include player camera views of a single player or multiple players. Similarly, the active viewer camera views presented to the passive viewers can include active viewer camera views of a single active viewer or multiple active viewers. The different camera views provided to the passive viewers are determined according to the viewing preferences determined from the user profile of the passive viewers. Thus, depending on what the viewers want to see, the camera views allow the passive viewers to watch different actions, thereby enriching the game watching experience of the viewers. The frames of the particular camera views identified for the passive viewers are forwarded to the game engine for processing, encoding, and streaming to the client devices of the passive viewers, where the streams are decoded and rendered.

[0055] The game engine (not shown) can include a multiplayer distributed game engine communicatively connected to the game logic of the game. Generally, a game engine is a layer of software used as the foundation for a game, such as an MMO game, and provides a framework for developing video games. The game engine abstracts the details of common related tasks needed for each game (i.e., game engine tasks), while the game developer provides game logic that provides the details of how the game is played. The game engine framework includes a number of reusable components for handling the several functional parts of a game (i.e., core features) that make a video game lively. Basic core features handled by the game engine can include physics properties (e.g., collision detection, collision response, trajectories, object movement based on gravity, friction, etc.), graphics, audio, artificial intelligence, scripting, animation, networking, streaming, optimization, memory management, thread handling, localization support, and so on. The reusable components include processing engines for handling the core features identified for the game.

[0056] During gameplay of the game, the game engine manages the game logic of the game, collects and transmits one or more player inputs received from one or more client devices 100 to the game logic. The game engine also manages the distribution and synchronization of the functional parts of the game engine to process the game data generated by the game logic in an optimal manner, and generates frames of game data that are transmitted back to the client devices 100 for rendering. Various game engines are currently available to provide different core functionalities, and an appropriate game engine can be selected based on the functions designated to be performed for the video game. In response to a request to view a game scene of the game, the game engine processes, encodes, and streams the recommended stream generated by the view recommendation engine to the client device of the spectator.

[0057] Game input provided by the players during game play corresponds to activities performed by the players in the video game. For example, activities performed by the players can cause certain actions to be performed in the game scene of the video game. The game input of the players, the activities performed, and the actions that occur in the video game are part of the telemetry data used to generate game play data 308. The game play data 308 and the telemetry data are stored in game play data storage 307. Game logic processes the game input provided by the players to affect the game state of the video game. The game state of the video game identifies the overall state of the video game at a particular point and is affected by the complexity of the game play of the video game affected by the game input of the players. If the video game is an MMO game, the overall game state of the video game is affected using input from multiple players. The game play data generated for game play includes saved data for each player. The saved player data includes any game customizations provided by the players for the video game. The telemetry data captures the characteristics of each activity that the players have attempted, the players have accomplished, the players have not accomplished, etc., the player attributes of the players, the game customizations of the players, game features, to name a few. The player attributes can be updated to the player profiles stored in user data storage 305. The game play data also includes image data related to the game scenes accessed during game play. The image data is used to generate the various camera views provided to the players, active viewers, and passive viewers.

[0058] Figure 2A Various camera views that can be generated for a game during game play are illustrated in one example implementation. Multiple players have accessed game cloud site 300 and selected a video game (Game 1) to play. Game play data is generated using the game input provided by each player. The game play data is processed to generate frames of player camera views that are forwarded to the respective players. One or more active viewers can select to follow the game play of one or more players. These active viewers provide input that is used to identify a particular camera view that captures each action that occurs in the video game. Figure 2AThe players PI - P6 are shown currently playing a game 1. Each of the players can be progressing through the game 1 at different paces. Each of the players is provided with a respective player camera view of the game scene in which actions are occurring in the game. The actions can occur in response to game inputs provided by one or more of the players or based on game logic of the video game. The player camera view (PCV) provided to each of the players shows the game scene of the game from the perspective of the player. Thus, player PI is presented with a player camera view PCVI showing the game scene of the game 1 from the perspective of the player PI, player P2 is presented with a player camera view PCV2, player P3 is presented with a player camera view PCV3, and so on. The game inputs provided by each of the players are transmitted to the game cloud site 300 via the network 200. The game logic of the game 1 executing on the game cloud site 300 uses the inputs of each of the players to affect the game state of the game 1 and generate game progression data. The game progression data resulting from the inputs of each of the players is processed to generate image frames that are transmitted back to the respective players as player camera views. The player camera views of each of the players illustrate the current game state of the game 1.

[0059] One or more active spectators have also logged into the game cloud site 300 to watch the game progression of one or more of the players. For example, spectators SI and S2 can have selected to watch the game progression of player PI, spectator S3 can have selected to watch the game progression of player P3, and spectator S4 can have selected to watch the game progression of player P5. Each of the active spectators actively provides inputs to specify a particular camera view that the active spectator wants of the actions occurring in the video game and generating the corresponding one or more player camera views. The game logic uses the game progression data corresponding to the time of occurrence of the actions and executes the video game again to generate an active spectator camera view for the active spectator according to the details included in the inputs. The inputs from the active spectators can identify a particular direction, depth, angle, clarity, etc. that the active spectator wants to be captured in the game scene associated with the respective action, and the active spectator camera view is generated accordingly. The view of the game scene captured by the active spectator camera view generated for each of the active spectators for each action is different from the view captured in the corresponding player camera view generated for the same action. For example, based on the inputs of the active spectator ASl, an active spectator camera view ASCVI.1 is generated of the game scene captured in the player camera view PCVI.1 of player 1 (i.e., an active spectator camera view of the game scene of player 1 that is different from the player camera view PCVI.1 of player 1), an active spectator camera view ASCV2.1 is generated of the game scene captured in the player camera view PCV2.1 of player 2 (i.e., an active spectator camera view of the game scene of player 2 that is different from the player camera view PCV2.1 of player 2), and so on. Figure 1Similarly, based on input provided by an active spectator AS2, an active spectator camera view ASCV1.2 is generated for the game scene captured in PCV1 of player 1 (i.e., active spectator camera view 2 of the game scene of player 1). Likewise, based on input provided by an active spectator AS3, an active spectator camera view ASCV3.3 is generated for the game scene captured in PCV3 of player 3, based on input of spectator AS4, an active spectator camera view ASCV5.4 is generated for the game scene captured in PCV5 of player 5, and so on. Each of the active spectator camera views generated from input of active spectators can be distinct and can identify different capture angles, capture depths, capture directions, capture clarity, etc. of the same game scene being viewed by the player. The generated active spectator camera views can show fewer or additional features of the game scene.

[0060] Figure 2B A breakdown of various camera views of a game scene in which action is occurring in game 1 is illustrated. The action can be in response to game input from player PI or based on game logic of the video game. Player PI is presented with a player camera view PCV1 that captures a view of game object 101 within the game scene. Two active spectator camera views have been generated for this game scene based on input provided by active spectators SI and S2. Thus, an active spectator camera view ASCV1.1 generated in response to input of active spectator SI shows a camera view of the game scene captured in PCV1 of player PI, but from a different view angle. Thus, ASCV1.1 includes a view of game object 101 included in PCV1 as well as a first game object 102 that is outside of the player camera view PCV1. This is because the active spectator camera view ASCV1.1 is captured beyond PCV1, as shown in FIG. 1 IB. Similarly, an active spectator camera view ASCV1.2 generated from input of active spectator S2 captures a different perspective of the game scene captured in PCV1 and includes a view of a second game object 103 that is outside of the game object 101 captured in PCV1. The active spectator camera view ASCV1.2 captures game object 101 from a different angle than captured in PCV1 and ASCV1.1. The active spectator camera views generated from input of different active spectators can capture game object 101 from different angles than captured in PCV. For example, one active spectator camera view (not shown) can capture a top-down camera view of game object 101, while another ASCV can show a view from an opposite side of game object 101 shown in PCV1. Game logic is configured to generate various views of each game scene in which action is occurring in game play based on input from different active spectators to capture different features of the game play. Figure 2B

[0061] ​When a passive spectator selects a video game to watch, the view recommendation engine executing at the game cloud site 300 is configured to extract different camera views generated for each action occurring in each game scene and select a particular camera view among the camera views for the action to present to the passive spectator. The particular camera view among the camera views is selected based on the viewing preferences of the passive spectator specified in the user profile of the passive spectator. The passive spectator simply selects a video game to watch and the view recommendation engine does the work of browsing through the different camera views generated for each action and identifies the appropriate camera view for the passive spectator. Each camera view presented to the passive spectator corresponds to an action occurring in a game scene. The camera view presented for each action lasts for the duration of the action captured within the camera view and is one of a player camera view or an active spectator camera view generated for the action.

[0062] The view recommendation engine dynamically switches between the player camera view and the active spectator camera view and the dynamic switching is based on the context of the action occurring in the respective camera view. For example, the video game can be a racing game and the camera views capture different views of the race. The player camera view PCV1 of player 1 captures the game scene of the race from the perspective of player 1 - showing player 1's car ahead of the competitors with no cars in front. The active spectator camera view of spectator S1 captures the same game scene and includes a second car associated with a second player that is coming from behind on the track and is trying to overtake player 1's car. The view recommendation engine evaluates the two camera views and determines that the action in the game scene captured by the active spectator camera view ASCV1.1 of AS1 (i.e., the relative position of the second car with respect to player 1's car in the race) is more compelling than the game scene captured in PCV1. Accordingly, the view recommendation engine can select ASCV1.1 of AS1 rather than PCV1 to present to the passive spectator.

[0063] In one embodiment, in addition to identifying a particular camera view based on the context of the action occurring within the respective camera view, the view recommendation engine can also filter the camera views according to the profile of the passive spectator. The profile of the passive spectator can specify that the passive spectator is interested in watching exciting actions or choosing an overhead view of the game scene or choosing a particular player or active spectator to focus on, and the view recommendation engine will identify the camera views according to the profile of the passive spectator. In the above example, if the passive spectator is interested in focusing on player PI, then the view recommendation engine can identify PCVI instead of ASCVI.I to present to the passive spectator. The passive spectator can just sit back and enjoy the curated camera views identified according to his viewing preferences. As the game play continues, the view recommendation engine dynamically switches between the PCVs of different players and the ASCVs of different active spectators, and the switching is correlated to the time of different actions occurring in the video game.

[0064] Figure 3A and Figure 3B FIG. illustrates different perspectives of the game play data stream within the game cloud site 300 for identifying camera views for a passive spectator. Camera views that capture different actions are streamed to the passive spectator, and the streaming of each camera view is correlated to different actions occurring within the video game. The camera views are part of the recommended stream provided to the passive spectator.

[0065] Figure 3A FIG. illustrates the game play data stream for generating various camera views of a video game in one embodiment, and Figure 3B FIG. illustrates the data stream for identifying Figure 3A a particular camera view among the camera views generated in FIG. and providing the particular camera view to the passive spectator. The various camera views are part of the recommended stream provided to the passive spectator.

[0066] Reference is made to Figure 3AIn one embodiment, multiple players access the game cloud site 300 via a network 200, such as the Internet, and request to play a video game (e.g., game 1). The request can include a game identifier for game 1. The game can be a multiplayer game played by the multiple players in the same location or remotely located. The game cloud site 300 receives and acknowledges the request from the players and, after successfully acknowledging each of the players, executes one or more instances of the game on one or more game consoles 303 / game servers 302 within one or more data centers 301. Game inputs provided by the multiple players during gameplay are processed by the game logic of game 1 and used to update the game state of game 1 and generate gameplay data 308. The game state of game 1 resulting from the game inputs of the players is synchronized across the instances of the game executed in the one or more data centers 301. In some embodiments, each player playing the game can be part of a team competing against other teams within the game. Another option is that each player can play the game independently of another player and as they progress within the game encounter different players in different game scenes of the game. In addition to the game inputs provided by the players, active spectators can also provide their own inputs to select a particular camera view they want to watch for each action occurring in game 1. The inputs of the active spectators do not change the game state of game 1. The inputs of the active spectators are processed by the game logic of game 1 to update the gameplay data 308 of game 1. The gameplay data 308 includes information used to generate different camera views for each action occurring in each game scene of game 1. For example, the gameplay data 308 is used to generate a plurality of PCVs 430a and a plurality of active spectator camera views (ASCVs) 430b. Each PCV 430a is generated for a particular player and in response to the game inputs provided by the respective player. The PCV identified for each player is forwarded to the respective client device of the player for rendering. Based on the rendered PCV, each player can provide further game inputs, which are processed by the game cloud site to generate further gameplay data. The view recommendation engine 400 uses the further gameplay data to train the recommendation model. As more gameplay data is provided to the recommendation model, the output of the recommendation model is fine-tuned. When a passive spectator requests gameplay data, the fine-tuning is used to identify better camera view recommendations for the passive spectator. Similarly, each ASCV 430b is generated for a respective active spectator based on the inputs provided by the particular active spectator. The ASCV 430b is fed back to the respective active spectator according to their specified view preferences defined in the inputs of the active spectator to allow the active spectator to watch the game scene in which the action is occurring during gameplay.

[0067] Reference is now made to Figure 3BA passive spectator PS1 interested in watching the game play of game 1 accesses game 1 executing on game cloud site 300 via network 200. Game cloud site 300 identifies the passive spectator using the passive spectator's login credentials and fetches the profile 309 of passive spectator PS1. The profile 309 of the passive spectator is provided as input to view recommendation engine 400. The profile 309 of the passive spectator includes the viewing preferences of the passive spectator collected over a period of time. The view recommendation engine also receives various camera views generated by game play data 308 of game 1. The view recommendation engine 400 uses a machine learning algorithm 410 to generate a recommendation model using information related to the camera views generated by the game play data as input, which is an artificial intelligence model. The recommendation model is further trained using game play data collected from ongoing game play sessions to generate different outputs matching different viewing targets specified by active spectators. The output is identified from the recommendation model to present to the passive spectator. The output from the recommendation model is used to select a particular camera view among the camera views of each action occurring in the video game. The camera view is selected based on the context of the action occurring in each camera view at the time of each action in the game scene of the video game. The identified camera view can be a player camera view or an active spectator camera view.

[0068] When more than one camera view is identified based on the context of the action, the view recommendation engine 400 further filters the camera views using the viewing preferences defined in the profile of passive spectator PS1. For example, two or more of the active spectator camera views can capture the view of the action occurring in the video game. In this example, the view recommendation engine 400 uses the viewing preferences of passive spectator PS1 to select one of the active spectator camera views to present to passive spectator PS1. The viewing preferences in the profile specify the viewing target of the passive spectator and are identified based on the viewing history of passive spectator PS1 collected over time, which can relate to game 1 and / or other games / interactive applications. The selected camera view that captures the view of the action occurring in the game scene of the video game is streamed to passive spectator PS1. The various camera views provided to passive spectator PS1 during the game play of the video game define a recommendation stream 430. The recommendation stream of passive spectator PS1 is specific to the passive spectator. Thus, different camera views can be provided to different passive spectators in their respective recommendation streams, and the camera views of different passive spectators are according to the context of the action occurring within the camera view and the viewing preferences of the respective passive spectator.

[0069] Next, the view recommendation engine 400 forwards the identified camera views in the recommended stream 430 for the passive spectator PS1 to the passive spectator's client device 100 for rendering. Thus, the view recommendation engine 400 is able to analyze the camera views of the game to identify specific camera views in the camera views of the game scenes of the game that are more meaningful to the passive spectator to enrich the passive spectator's game viewing experience.

[0070] The request from the passive spectator PS1 can be to watch the current live-streamed game play or a pre-recorded game play of the game played by the plurality of players. For simplicity, various embodiments are discussed in terms of a current live-streamed game play session of a game, but the various embodiments can also be extended to a pre-recorded game play session. In addition, the various embodiments can be applicable not only to MMO games, but also to content generated in real-world environments, whether live-streamed or pre-recorded for later streaming, live direction of a movie production, etc.

[0071] The details of the identified camera views for the passive spectator PS1 are forwarded to a stream recommendation engine (or simply "recommendation engine") 420. The recommendation engine 420 uses the selected camera views for each action in each game scene of the game to generate a recommended stream 430. The generated recommended stream 430 is an aggregation of the selected camera views for different actions in different game scenes of the game. The identified camera views in the recommended stream 430 are forwarded to the client device of the passive spectator PS1 via the network 200 for rendering. The camera views forwarded to the passive spectator PS1 can be a combination of one or more player camera views that capture different views of the different actions and / or one or more active spectator camera views. Providing the selected camera views in the camera views that capture the game scenes in which the actions occur based on the context of the actions and, where appropriate, according to the viewing preferences of the passive spectator PS1 allows the passive spectator to be more immersed in the game, thereby enhancing the passive spectator's interest in the game.

[0072] Figure 4FIGURE 1 illustrates details of a view recommendation engine 400 for processing game play data 308 generated during game play of a game in one implementation. As previously described, the game can be a multi-player game played by a plurality of players. Game inputs provided by the plurality of players of the game are used to update the game state of the game and generate game play data 308. The game play data 308 is used to generate different game camera views that are part of a recommended stream 430 identified for passive observers of the game. The game play data includes game inputs 308a of each of the players. The game inputs 308a of each of the players are used to affect the outcome of the game. In addition to the game inputs 308a provided by the players, one or more active observers can provide inputs 308b during game play to affect generation of an active spectator camera view (ASCV) 430b. In response to the inputs 308b from the active observers, the game logic executes the game again using the game play data 308 available at the time the inputs 308b are received from the active observers to generate the ASCV 430b of the action occurring in the game scene. The different PCVs 430a captured for each action occurring in the game scene capture different player perspectives of the action, while the ASCV 430b generated for each action captures a different view and aspect than the view and aspect captured in the corresponding PCV 430a. The view of each action in the game scene captured by the ASCV 430b as well as the PCVs 430a includes sufficient data to enable construction of a three-dimensional representation of the game scene associated with the action.

[0073] To identify the camera views that are part of the recommended stream 430 passed back to passive observers of the game 1, the game play data 308 is processed to extract image data 311 and telemetry data 312. The image data 311 provides details of various image features included in each game scene accessed by one or more players in the video game. The telemetry data 312 captures details of the game play, including characteristics of each activity of the game inputs provided by the players, player attempts, player achievements, player unaccomplishments, player attributes of the players, effects of the game inputs of the players on the game state, game features of the game (including genre, game play level, complexity), etc. The image data 311 and the telemetry data 312 are forwarded to a feature processing engine 401 for further processing. The image data 311 can be stored in an image data storage 311a, and the telemetry data 312 can be stored in a telemetry data storage 312a.

[0074] The feature processing engine 401 extracts the image data 311 from the image data store 311a and processes the image data 311 to extract various features of the images captured in each camera view of the game play data. The image features of the game scene can include features (e.g., length, width, height, size, weight, color, static or dynamic characteristics, etc.) of each game object included in the game scene, such as the number and location of each game object; the number, location, and features of non-playing characters; the number, location, and features of the player's adversaries or partners; and so on. In addition, the image features can also include the angle of capture, the depth of capture, the clarity of the captured images, and so on. The feature processing engine 401 also extracts the telemetry data 312 from the telemetry data store 312a and processes the telemetry data 312 to extract features of the game play and attributes of the player providing the game inputs that affect the game play. The game play details included in the telemetry data 312 can be used to identify the complexity of various game inputs provided by the player within the game scene captured in each camera view and the results of the game inputs.

[0075] The image features, game features, and player attributes extracted from the game play data 308 of the game play period of the game are provided as inputs to the machine learning algorithm 410. The machine learning algorithm includes classifiers 410a defined for different features and attributes extracted from the game play data of the game. The classifiers 410a are used to generate a recommendation model 410b. The recommendation model 410b is trained using the game inputs received from the player, additional game inputs collected over time during the game play of the video game. In one implementation, the game inputs are collected from multiple game play sessions. The recommendation model is trained to fine tune the attributes included in various nodes of the recommendation model, the features, and to strengthen the relationships defined by the edges between pairs of consecutive nodes of the recommendation model. Various outputs are defined in the recommendation model, where each output is related to a specific game behavior of the player and / or a viewing preference of the active spectator. The output from the recommendation model 410b is selected based on the context of the action appearing in each of the various camera views that captured the respective action. The selected output identifies the camera view that will be provided to the passive spectator in response to a viewing request from the passive spectator. In addition to the context of the action used to select a specific camera view, the viewing preference of the passive spectator can also be used to identify the output from the recommendation model. The viewing preference of the passive spectator can be obtained from the profile 309 of the passive spectator (which can host the game types, action types, viewing types, etc. that the passive spectator likes), and this information can be obtained from the viewing history of the passive spectator gathered over a period of time. The camera view is identified and passed back to the passive spectator to satisfy the viewing preference of the passive spectator.

[0076] In some implementations, one or more camera views of the captured action are identified that are in accordance with the context of the action and the viewing preferences of the passive spectator. In the described implementations, the viewing preferences of the passive spectator are weighted and the camera views are further filtered based on the weighted preferences to identify a particular camera view of the camera views that best matches the weighted preferences of the passive spectator. The particular camera view of the camera views is recommended to the passive spectator. The camera view recommended to the passive spectator is in accordance with the action occurring in the game scene and in some instances in accordance with the viewing preferences of the passive spectator to allow the passive spectator to have an optimal viewing experience. The output from the recommendation model 410b is provided to a recommendation engine 420 that identifies a curated camera view of the camera views for each action. The camera view selected for an action is provided for the duration of the action occurring in the game scene and after the action is completed, the camera view is dynamically switched to a different camera view that corresponds to the action occurring at the time of the dynamic switch. The camera views streamed to the passive spectator for different actions define a recommended stream 430 of content for the game play. The recommended stream 430 includes one or more PCVs 430a and one or more ASCVs 430b. The series of camera views in the recommended stream match a series of various actions occurring in the game play of the video game. In one implementation, the recommended stream 430 generated by the recommendation engine 420 can be specific to each passive spectator as the recommended stream can include one or more camera views in accordance with the viewing preferences of the respective passive spectator. The recommended stream 430 for the passive spectator is streamed back to the respective client device 100 of the spectator for rendering in response to a request from each passive spectator.

[0077] Figure 5 FIG. illustrates various modules of the view recommendation engine 400 in one implementation for processing game play data of a game to generate a recommended stream for each passive spectator that has requested game play data of the game. The view recommendation engine 400 includes a plurality of modules / processing engines to process the game play data and identify various features and attributes included in the game play data. The modules included in the view recommendation engine 400 can include a telemetry feature extraction engine 402, an image feature extraction engine 403, a player attribute extraction engine 402a, a spectator attribute extraction engine 402b, and a game play data extraction engine 402c. In addition to the aforementioned modules / engines, the view recommendation engine 400 can also include a feature processing engine 401, machine learning algorithms 410 including a classifier (i.e., classification engine) 410a, a recommendation model 410b, and a recommendation engine 420 to name a few.

[0078] Game inputs generated at the respective client devices of the players during gameplay of the game are forwarded to one or more game servers 302 executing an instance of the game for further processing. Game logic of the game uses the game inputs to update the game state of the game and generate gameplay data 308. The gameplay data 308 is used to generate a plurality of camera views as part of the recommended stream 430 fed back to the passive spectator. The plurality of camera views capture different perspectives of the actions occurring in different game scenes of the game. The camera views generated from the gameplay data 308 include player camera views (PCV) that capture the perspective of the respective player for each action occurring in each game scene of the game. In addition to the PCV, a plurality of active spectator camera views (ASCV) are generated using inputs provided by the active spectator. The inputs from the active spectator specify the depth, direction, level of clarity, angle, content that the active spectator wants to capture for each action. The game logic uses the inputs of the active spectator to execute the video game again to generate the active spectator camera view (ASCV) for each action. The PCV 430a and ASCV 430b, along with the gameplay data 308, are forwarded to a view recommendation engine 400 executing at the game server 302. The game server 302 executing the view recommendation engine 400 can be the same game server 302 executing the instance of the game accessed by the one or more players and / or spectators, or can be a different game server 302 communicatively connected to the one or more game servers 302 executing the game. The players can participate in the gameplay of the game by accessing a single instance of the game executing at the game server 302. Another option is that the players can access different instances of the game executing at different game servers 302. In that case, the game state of the game is synchronized across all executing instances of the game so that the current game state of the game can be presented to the players.

[0079] The communicative connection enables the view recommendation engine 400 to receive the gameplay data 308 and the generated game camera views as camera streams. The gameplay data 308 captures telemetry data 312 and the game camera views capture image data 311 of the gameplay of the plurality of players. The telemetry data 312 captures the complexity of the actions performed in the game based on the game inputs from the players.

[0080] The telemetry feature extraction engine 402 is configured to process the telemetry data 312 contained in the game camera view. The telemetry data 312 includes player attributes defined based on game inputs and gameplay skills of each player and game features resulting from execution of the game. The telemetry feature extraction engine 402 includes a player attribute extraction engine 402a, a spectator attribute extraction engine 402b, and a gameplay data extraction engine 402c. The player attribute extraction engine 402a is configured to extract player related data from the telemetry data 312. Some of the player related data that can be extracted includes data related to player ratings, player level, game progress, gameplay strategies taken by each player, entertainment values related to gameplay and related to players, player movement sequences, unique moves performed by players, etc. The player rating data of each player can be updated in accordance with rating inputs provided by other players, active spectators, content providers, etc. The player related data is used to determine player attributes (e.g. popularity, skill / professional level, etc.) of each player providing game inputs in the game. The player related data can be stored in the user data store 305 and can be used to update player profiles included in the user accounts 304.

[0081] The gameplay data extraction engine 402c is configured to extract data related to game features (such as game level data), data related to activities performed by players (including attempted activities, completed activities, number of times each activity was attempted), game win data of each player, game state data, etc. The activities performed by the players through game inputs can be related to actions performed in the game scene of the game. The game related features are used to determine game state, save data related to each player (including game customizations performed by each player), etc. The game related features can be stored in the gameplay data store 307.

[0082] The spectator attribute extraction engine 402b is configured to extract active spectator related data from the telemetry data 312. Some of the active spectator related data that can be extracted includes the number of active spectators paying attention to the game play of the game, the number of active spectators paying attention to each of the players, the profiles of the active spectators including the spectator ratings, and the inputs provided by the active spectators. The inputs provided by the active spectators can include camera view related inputs that capture the game actions that occur in the game scene of the game, such as the specific angle, depth, direction, etc. In addition to the foregoing inputs, the active spectators can also provide other game related inputs related to the game play of one or more players, such as comments, ratings, chats, emails, blogs, etc. The game related inputs can be used to identify the type of comments provided, the quality of the comments, the audience awareness of the game, etc. The active spectator related data can be updated to the profiles of the spectators and can also be used to update the game features and / or provide game inputs to the player attributes (e.g., popularity, skill level, etc.) of each of the players of the game. The active spectator related data can be stored in the user data store 305 separately from or together with the player attributes.

[0083] The image feature extraction engine 403 is similarly configured to extract various image features from the image data 311. The image features extracted from the image data 311 can be used to distinguish the PCV 430a from the ASCV 430b within the view of the game play. The extracted image features can include data related to the capture angle, the capture depth, the capture direction, the clarity of the images captured in each of the capture views, the content captured in the images, the game objects targeted by the game inputs from the players, the location and features of the game objects targeted by the game inputs, etc. The image related data can be stored in the image data store 311a.

[0084] The player attributes extracted by the player attribute extraction engine 402a, the active spectator attributes extracted by the spectator attribute extraction engine 402b, the game features extracted by the game play data extraction engine 402c, and the image features extracted by the image feature extraction engine 403 are provided as inputs to the feature processing engine 401 for further processing. The feature processing engine 401 identifies the complexity of the game play and correlates the features identified from the various camera views generated from the game play data, the attributes with the specific actions occurring in each game scene of the game. The features, attributes, actions, game scenes, and other game related data processed by the feature processing engine 401 are provided as inputs to the machine learning algorithm 410. The machine learning algorithm 410 includes a classifier 410a that is defined using one or more image features, game features, player and / or active spectator attributes. A recommendation model 410b is generated using the information included in the classifier 410a. The recommendation model 410b is an AI model that is built and trained using various features and attributes extracted from the game play data 308. The recommendation model 410b includes a plurality of nodes and edges. The features, attribute information defined in the various classifiers are used to populate the various nodes of the recommendation model 410b and the edges between any pair of consecutive nodes are used to define the inter-relationships between the information contained in the respective nodes. As additional game data is generated from the game play of the game, the AI model is trained with the additional game data. The additional game data is used to fine tune the features, attributes included in the respective nodes and the corresponding edges defining the inter-relationships between the respective nodes so that the inter-relationships are stronger. The machine learning algorithm 410 identifies various camera views to present to the passive spectators interested in watching the game play of the game. The various camera views identified can include one or more of the player camera views or the active spectator camera views and the camera view selected for the passive spectator is based on the context of the actions occurring in the relevant player camera views and active spectator camera views.

[0085] The machine learning algorithm 410 also receives the profiles 309 of passive viewers interested in watching the game play of the game. The profiles 309 of the passive viewers identify viewing preferences of the passive viewers obtained from the viewing history of the passive viewers. These viewing preferences define the viewing targets of the passive viewers. In some implementations, the machine learning algorithm 410 can also use the viewing preferences of the passive viewers to identify the outputs of the AI models that best match the viewing targets of the passive viewers. Using the outputs from the AI models to identify various camera views that match the preferences of the viewers, in addition to identifying the camera views that are based on the context of the actions captured in the respective camera views. The recommendation model can identify different outputs for different passive viewers, with each output identifying a set of camera views of the game scene of the game that capture the actions occurring in the game. The set of camera views are further filtered to identify particular camera views in the camera views that match the preferences of the particular passive viewers. The outputs from the AI models are provided to the recommendation engine 420 as inputs. The recommendation engine 420 uses the outputs from the AI models to extract data related to the curated camera views of the camera views, processes the data to generate frames of the game scene content and streams the frames of the game scene to the client devices 100 associated with each of the passive viewers via the network 200 for rendering. The camera views provided to the passive viewers are part of a recommended stream that is streamed to the passive viewers during the game play of the game and provides views of the actions that are consistent with the content that the passive viewers wish to watch.

[0086] Figure 6FIG. 1 illustrates various modules within a feature processing engine in one embodiment for processing various features of a game play and attributes of players extracted by respective extraction engines. In one embodiment, the feature processing engine 401 includes, for example, a telemetry feature processing engine 404, an image data analyzer 405, and a score generation engine 406, to name a few. The telemetry feature processing engine 404 processes telemetry data 312 extracted from the game play data 308 provided by the game server 302. The telemetry feature processing engine 404 can include multiple sub-modules for processing different telemetry data 312. For example, a player attribute processing engine 404a within the telemetry feature processing engine 404 can be used to process data related to player attributes extracted by the player attribute extraction engine 402a and stored in the user data store 305. The player attributes in the user data store 305 are updated when additional player attributes of a player are detected or changes to existing player attributes are detected. The player attributes can be used to determine the game play style of a player in the game, the popularity of a player, the game play value of a player (e.g., entertainment value, strategy value, professional level of a player, etc.), and game customizations defined by a player. Similarly, a spectator attribute processing engine 404b within the telemetry feature processing engine 404 can be used to process spectator attributes of active spectators extracted by the spectator attribute extraction engine 402b and stored in the user data store 305. Active spectators are those spectators who actively provide input to specify the camera view of different actions they want to watch during the game play. The spectator attributes can be used to define or update additional attributes of a player and / or additional game features of the game.

[0087] A game play data processing engine 404c within the telemetry feature processing engine 404 can be used to process game features extracted by the game play data extraction engine 402c. The game features can be used to determine the game state, game save data for each player, the popularity of the game among players and / or active spectators, game complexity, etc.

[0088] The image data analyzer 405 is configured to analyze the image data 311 extracted by the image feature extraction engine 403 and stored in the image data storage 311a to identify various features of the game scenes of the game. The features can be related to the content of the game scene, the actions occurring in the game scene, one or more activities performed by different players in the game scene, and / or the quality / complexity of the game scene. The image data analyzer 405 can include a plurality of sub-modules for evaluating various features included in the image data 311. The evaluation of the image data includes giving a relevance score for each of the image features identified and using the given scores in defining the classifier 410a by the machine learning algorithm 410. Some of the sub-modules of the image data analyzer 405 include a content evaluation engine 405a, a rating evaluation engine 405b, and an activity evaluation engine 405c for evaluating the features of the image data 311. The content evaluation engine 405a is configured to evaluate the image data to identify the content captured within each game scene. The content can include game objects, non-player characters, scenes, challenges, adversaries, partners, etc. that provide a panorama of the game scene. The image data related to the features can include feature identifiers, modifying characteristics such as height, weight, length, width, depth, size, appearance, location, physical characteristics (e.g., moving or non-moving, direction of movement, trajectory, speed, etc.), etc. The rating evaluation engine 405b of the image data analyzer is configured to evaluate the popularity of the game scenes of the game based on the number of times the registered players browse the game scenes during the game play, the number and type of comments received from the active audience related to the features identified in the game scenes, etc. The activity evaluation engine 405c of the image data analyzer is configured to evaluate the various actions occurring in each game scene, the activities performed by the players in each game scene through the game inputs. The evaluation also determines the frequency of each activity occurring in each game scene, the type and number of game inputs provided to perform each activity, the type of action, the duration of the action, etc. The rating evaluation engine 405b and the activity evaluation engine 405c evaluate the image data derived from the game inputs (e.g., activities, actions, etc.) from the players and the inputs from the active audience (e.g., comments, chats, ratings, types, and frequency of interactions of the active audience, etc.). The inputs from the active audience can be used to update the profiles of the corresponding players and / or the game features of the corresponding game scenes. The activity evaluation engine 405c is configured to identify and evaluate both the player-related activities and the active audience-related activities.

[0089] Information from the telemetry feature processing engine 404 and the image data analyzer 405 is provided as input to the score generation engine 406. The score generation engine 406 is configured to generate relevance scores for different camera views based on the identification and evaluation of various features and attributes. For example, a content-based score 406a can be computed based on the evaluation of the content captured in each camera view of the game scene. Similarly, an activity / action-based score 406b for each camera view can be computed based on the activities performed by the player and the actions that occur in each game scene captured in the respective camera view in response to the activities. A player-based score 406c for a camera view can be computed based on the player attributes determined from the activities performed in the game scene captured in the respective camera view. An audience-based score 406d for different camera views can be computed based on the input provided by the active audience for each camera view of the game scene. The scores from the score generation engine 406 are associated with the respective camera views and used during the generation and tuning of the recommendation model (i.e., the Al model) 410b. The scores for the various camera views of different game scenes are continuously updated based on the game inputs received from the player and the input from the active audience.

[0090] The input provided by the active audience specifies the game viewing preferences for each action that occurs in the game. The game viewing preferences for a game scene can be used to capture the actions that occur within the game scene and can specify the angle, depth, direction, etc. of the capture of the actions. These viewing preferences can be specific to each action and can be based on the type of action that occurs in the game scene. The game viewing preferences define the viewing target of the audience and are used to generate the specific active audience camera view of the game.

[0091] The recommendation model 410b is fine-tuned using the game inputs of the players and the inputs of the active audience collected for the game. The fine-tuning helps to reinforce the various features, inter-relationships between the attributes identified from the game play data. The recommendation model 410b identifies different outputs for different viewing objectives. When a passive audience selects a game to watch, the output that satisfies the viewing objective of the passive audience is identified from the recommendation model 410b. The viewing objective can be based on the context of the action occurring in the game scene and in some cases also based on the profile of the passive audience. The audience profile identifies the type of action, the amount of detail of the action, the type of camera view (top view, side view, etc.) preferred by the passive audience when watching the game, etc. The viewing preference details of the passive audience are gathered from the viewing history of the passive audience, which can be related to the game application and / or any other interactive application. The output from the recommendation model 410b is used to identify the camera view of each action occurring in the game scene to present to the passive audience. For example, based on the action occurring in the game scene, one or more camera views can be identified to present to the passive audience. When more than one camera view is identified, the viewing preference of the passive audience can be used to identify the camera view of the action. The camera view identified for the passive audience can be the same or different from the PCV. For example, the profile of the passive audience can show that the passive audience generally likes to watch the top camera view of the action, and the appropriate top camera view of the action can be presented to the passive audience to watch the action. In another example, it can be desirable to present different camera views of different types of action to the passive audience, such as a top view for a racing game, a side view for a basketball game, an opponent view for a tennis game, etc. In one implementation, the opponent view can be defined as a view that captures the action (e.g., hitting or receiving a ball) from the perspective of the opponent player (such as the opponent in a tennis game), while the PCV of the same action can capture a view from the perspective of the player looking at the ball moving towards the opponent player.

[0092] In one implementation, the active spectator camera views generated from the input from the active spectator can be direction-based camera views or action-based camera views. The score generation engine 406 computes various feature / user-based scores from the type of the generated ASCV. The scores computed for different camera views are provided to the machine learning algorithm 410 to further tune the recommendation model 410b so that appropriate camera views can be identified for presentation to the passive spectator. Different camera views are presented to the passive spectator during the game play of the video game, where each camera view corresponds to a change in action that occurs in the game scene and is presented to the passive spectator for the duration of the change. The camera views presented to the passive spectator are dynamically switched to correspond to the action that occurs in the game scene. The dynamic switching is carried out without any input from the passive spectator and is designed to provide the passive spectator with camera views that capture various actions that occur in the game according to the action that occurs and according to the profile of the passive spectator. The camera views presented to the passive spectator are part of the recommended stream 430.

[0093] The recommendation algorithm can provide different camera views to different passive spectators in the recommended stream 430 of the different passive spectators even when the same action occurs in the game scene. The difference in the camera views of one passive spectator from another passive spectator can be due to different viewing preferences specified in the respective profiles of the passive spectators. The recommendation model 410b is configured to identify appropriate outputs with the viewing preferences of the different passive spectators in consideration. The outputs identified from the recommendation model 410b are forwarded to the recommendation engine 420. The recommendation engine 420 uses the information provided in the outputs to identify a particular camera view among the camera views to pass back to the passive spectator. When the camera view is in line with the viewing preferences of the passive spectator, the passive spectator gets a satisfactory game viewing experience based on the camera view presented to the passive spectator.

[0094] Figures 7A-1 to 7E-1 FIG. illustrates sample switched camera views provided by the recommendation engine 420 to multiple passive spectators of a game based on the outputs identified from the recommendation model. Figure 7A-1The illustration shows a sample camera view switching provided to passive viewer 1. The camera view switching includes a combination of the player camera view (PCV) and the active viewer camera view (ASCV). For example, the recommended camera view switching provided to passive viewer PS1 includes the initial player's PCV (player identifiers 35, 731), followed by ASCV16.1, PCVs for players 3 and 42, ASCV20.7, and the PCV for player 35. Various camera views are provided during video game rendering by dynamically switching existing camera views to new ones in essentially real-time. Each camera view is rendered for a period of time before the switch, where the time period is related to the duration of the action appearing in the corresponding camera view. For example, PCV35 is rendered for 4.5 seconds, then dynamically switched to and replaced by PCV731. During gameplay, PCV731 is rendered from 4.5 seconds to 7.5 seconds up to 12 seconds, then switched to and replaced by ASCV16.1. Rendering ASCV16.1 takes 7 seconds, then it switches to PCV3, and so on. Figure 7A-1 In the exemplary recommended camera view shown, PCV35 is shown twice—once at the beginning of the sample and once at the end. This instructs the camera view of player 35 to capture image features and / or views of ongoing actions that the passive viewer PS1 likes to see, and in some instances, it can also be matched with the passive viewer PS1's preferences at different times. The ASCV presented to the passive viewer identifies the game scene of the player that the active viewer is interested in and is generated based on the active viewer's input. For example, ASCV16.1 corresponds to the active viewer's camera view. Figure 1 The active viewer camera Figure 1 The system captures game actions within the game scene associated with player 16. Similarly, ASCV20.7 corresponds to active viewer camera view 7, which captures game actions within the game scene associated with player 20's game scene. Figure 7A-2 This shows the generated camera view stream provided to a passive viewer on a PS1 during the gameplay of a video game.

[0095] Figure 7B-1 The illustration shows another example of a sample switching camera view provided to a second passive viewer (PS2) in one implementation. The recommended stream includes a set of ASCV 3.5, 73.1, and 16.1 at the beginning, followed by PCV for players 3 and 42, ASCV 20.7, PCV for player 3, and ASCV 3.5. (As in...) Figure 7A-1As in the first example, certain camera views in the camera view sequence repeat. The switching camera views generated for each passive spectator can capture not only the game scene of the game from different angles, but also the progression of the action occurring in the game scene. The action occurring in the game scene can or can not be in response to game input from the player. Figure 7B-1 One such example is illustrated, in which the initial ASCV 3.5 can capture the beginning of the action and the same ASCV 3.5 at the end can capture the progression of the action in the game scene over time. In this example, the initial and end ASCV 3.5 can capture the game scene of the action from the same angle, while other ASCVs and PCVs can capture the game scene associated with the action from different angles (including different heights, different depths, different modes, different directions, etc.). Figure 7B-2 A generated camera view recommendation stream is shown provided to passive spectator PS2 during viewing of the game play of the video game.

[0096] Figure 7C-1 Another example switching camera view provided to passive spectator PS3 is illustrated, which includes a PCV of player 35, followed by a string of ASCVs (ASCV 35.7, ASCV 6.1, ASCV 318.3, ASCV 42.8, ASCV 20.7, ASCV 35.9) and a PCV 35. In one implementation, all of the camera views included in the switching camera view provided to PS3 can capture the same game scene associated with one action or multiple actions, and each camera view can present the action from a different perspective according to the viewing preferences of the spectator. Figure 7C-2 A generated camera view recommendation stream is shown provided to passive spectator PS3 during viewing of the game play of the video game.

[0097] Figure 7D-1 Another example switching camera view provided to passive spectator PS4 is illustrated, which includes only PCVs of different players that capture views of one or more actions occurring in the game scene of the game. In this example, the PCV of player 35 is shown at the beginning and end of the switching camera view. This can show the progression of the action occurring within the game scene. Figure 7D-2 A generated camera view recommendation stream is shown provided to passive spectator PS4 during viewing of the game play of the video game.

[0098] Figure 7E-1 Another example is illustrated in which the switching camera view provided to passive spectator PS5 includes only ASCVs generated from input of an active spectator, which capture one or more actions occurring in the game scene. Figure 7E-2A generated camera view recommendation stream is shown being provided to a passive spectator PS5 during viewing of gameplay of a video game. The foregoing example is provided as an illustration to demonstrate that the camera views identified for a passive spectator based on the passive spectator's specified preferences can include any combination of camera views that capture different actions or different views of the same action.

[0099] Various implementations described herein provide ways to enhance a viewing experience of a passive spectator by identifying and presenting a curated set of camera views of a game scene of a game play generated from a plurality of players' game plays. The curated set of camera views are identified based on the passive spectator's preferences specified in the passive spectator's profile rather than being recommended based on the game's game logic, which can provide camera views from the players' perspectives or based on popularity of actions or popularity of camera views. Providing the passive spectator with camera views relevant to the passive spectator of the game (i.e., in terms of what the passive spectator wants to see) can result in a satisfactory viewing experience for the passive spectator, which can result in viewers staying in the game. Viewer stay can translate into more revenue for game developers, game players, game sponsors, etc.

[0100] Figure 8 An operational flow of a method for improving a viewing experience of a spectator in one implementation is illustrated. The method starts with executing a video game at a game server, as illustrated in operation 810. The execution of the video game can be in response to a request for a game play of the video game received from one or more players. The execution of the video game causes game play data to be generated from game inputs provided by the players during the game play. The game play data is used to generate camera views, which include a player camera view for each of the players and an active spectator camera view for each active spectator accessing the video game to view the game play of the players. The active spectator camera view is generated based on active inputs received from the active spectator, where the inputs can specify viewing preferences of the active spectator. The viewing preferences of the active spectator can include a capture angle, a capture depth, a capture direction, a capture clarity, etc. Thus, the active spectator camera view generated for each active spectator is also referred to as an active spectator camera view. The game can be a multi-player game played by a plurality of players in the same location or in remote locations in different geographical locations. The game inputs provided by each player are used to affect a game state of the game and generate game play data. The game play data is used to generate a plurality of camera views that capture different aspects of a game scene of the game. The various camera views generated for the game scene can be used to generate a three-dimensional view of the game scene. The camera views generated from the game play data include content frames that capture views of one or more actions occurring in the game scene. The content frames associated with the player camera view of a player are streamed live to the player.

[0101] A request is received from a passive spectator to view gameplay of a video game, as illustrated in operation 820. The request specifies a video game that the passive spectator is interested in viewing, but does not include any other input that affects the generation of camera views. In response to the request from the passive spectator, a view recommendation engine executing on the game server identifies a player camera view and an active spectator camera view that capture an action occurring in a game scene of the video game to present to the passive spectator in response to the request, as illustrated in operation 830. The player camera view and the active spectator camera view related to the action are generated using game play data available at the time the action occurs. The view recommendation engine then dynamically switches between the player camera view and the active spectator camera view based on a context of the action occurring in the player camera view and the active spectator camera view. The dynamic switching is performed automatically without any input from the passive spectator. The camera view provided to the passive spectator is dependent on the type of action being captured. In some embodiments, the camera view is also based on viewing preferences of the passive spectator gathered from a viewing history included in a profile of the passive spectator.

[0102] Traditionally, game logic would present all camera views generated for each action in each game scene, and the passive spectator would have to browse through each camera view to identify a camera view that captures the game scene according to their preferences. Or alternatively, game logic would present a player camera view of a particular player by matching the player's profile with the passive spectator's profile. The camera view can not be optimal for the passive spectator because the passive spectator can be interested in viewing a different perspective of the game scene where the action occurs, rather than the perspective provided by the game logic. The passive spectator can not be interested in providing input like the active spectator to generate a new camera view, but can be interested in viewing a camera view that best captures the action within the game scene.

[0103] Various embodiments of the present disclosure discussed herein provide camera views of different actions occurring in a game to a passive spectator by using camera views that have already been generated for players and active spectators, without having to form a passive spectator's own camera view. The camera views provided to the passive spectator match the passive spectator's viewing preferences and provide a view of the action within the game scene to enable the passive spectator to have a satisfactory game viewing experience.

[0104] In some implementations, a user interface can be provided to a passive spectator to update viewing preferences in a profile of the passive spectator. The viewing preferences specify viewing goals of the passive spectator. A view recommendation engine cooperates with game logic to select a camera view according to the viewing goals of the passive spectator. The user interface can include options for the passive spectator to select. The various implementations discussed herein can be used for live-streamed games and pre-recorded game play of games.

[0105] In addition to game play of games, the various implementations can be extended to real world content that is captured and streamed to spectators. In one implementation, the real world content can be captured by a plurality of users, where the users can be grouped into different teams. Each team of users can capture the real world scene from different angles and forward the captured stream to an application cloud server to share with other users / spectators. The application cloud server associated with the view recommendation engine can process the content of the captured real world scene to identify various camera views that capture actions in the real world scene and identify a select camera view among the camera views to stream to a passive spectator based on preferences of the passive spectator. As with game play of games, the real world scene captured by the plurality of users can be live-streamed to the passive spectator via the application cloud server, or another option is that a recording of the real world scene can be stored in the application cloud server and used when the passive spectator desires to watch a pre-recorded video of the real world scene. In the implementation of capturing real world content, an active spectator can be a spectator that provides input to influence the content captured in the camera view. The input can be in the form of feedback or instructions to the users that capture the real world content. An alternative implementation can include receiving real world content captured by a plurality of users, where each user captures the real world content from a different angle / depth / direction, and using input of an active spectator to identify a select camera view among the camera views to present to a passive spectator. The passive spectator only provides a request to view the real world content, and the view recommendation engine will automatically identify a particular camera view among the camera views to provide to the passive spectator. The passive spectator can sit back, relax, and enjoy the camera view of the game play that is selected for the passive spectator as the optimal camera view for the passive spectator.

[0106] Figure 9An embodiment illustrating an information service provider architecture. Information service providers (ISPs) 902 deliver a multitude of information services to users 900 that are geographically dispersed and connected via a network 950. An ISP can deliver only one type of service (such as stock price updates) or a variety of services (such as broadcast media, news, sports, games, etc.). Additionally, the services provided by each ISP are dynamic, i.e., the services can be added or taken away at any point in time. Thus, the ISP providing a particular type of service to a particular individual can change over time. For example, a user can be served by an ISP close to the user when the user is at home, and can be served by a different ISP when the user travels to a different city. The home ISP will transfer the required information and data to the new ISP so that the user information "follows" the user to the new city, making the data closer to the user and more easily accessible. In another embodiment, a master-server relationship can be established between a master ISP that manages the information for the user and a server ISP that interfaces directly with the user under the control of the master ISP. In another embodiment, when a client moves around the world, data is transferred from one ISP to another so that the ISP that is in a better location to serve the user becomes the ISP that delivers these services.

[0107] The ISP 902 includes an application service provider (ASP) 906 that provides computer-based services to customers via a network (e.g., including, for example and without limitation, any wired or wireless network, LAN, WAN, WiFi, broadband, cable, fiber, satellite, cellular (e.g., 4G, 5G, etc.), the Internet, etc.). Software provided using the ASP model is sometimes also referred to as on-demand software or software as a service (SaaS). A simple form of providing access to a particular application, such as customer relationship management, is to use a standard protocol such as HTTP. The application software resides on the vendor's system and is accessed by the user using HTML through a web browser, through specialized client software provided by the vendor, or through other remote interface such as a thin client.

[0108] Services provided over a wide geographic area often use cloud computing. Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet. Users do not need to be experts in the technology infrastructure in the "cloud" that supports them. Cloud computing can be divided into different services, such as infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). Cloud computing services often provide online general business applications accessed from a web browser, while software and data are stored on servers. Based on the way the Internet is depicted in computer network diagrams, the term cloud is used as a metaphor for the Internet (e.g., using servers, storage, and logic), and is an abstraction of the complex infrastructure it conceals.

[0109] In addition, the ISP 902 includes a game processing server (GPS) 908 that is used by game clients to play single-player and multiplayer video games. Most video games that are played via the Internet operate via a connection with a game server. Typically, the game uses a dedicated server application that collects data from the players and distributes the data to other players. This is more efficient and more effective than a peer-to-peer arrangement, but requires a separate server to host the server application. In another implementation, the GPS establishes communication between the players and their respective game-playing devices to exchange information without relying on a centralized GPS.

[0110] A dedicated GPS is a server that runs independently of the client. Such a server typically runs on dedicated hardware located in a data center, providing greater bandwidth and dedicated processing power. For most PC-based multiplayer games, a dedicated server is the preferred method of hosting a game server. Large multiplayer online games run on dedicated servers, usually hosted by the software company that owns the game, allowing the dedicated server to control and update the content.

[0111] A broadcast processing server (BPS) 910 distributes audio or video signals to an audience. Broadcast to a very narrow audience is sometimes called narrowcasting. The last leg of broadcast distribution is how the signal gets to the listener or viewer, and it can come over the air as with a radio or television, or can come through a cable television or cable radio (or "wireless cable") directly to a receiver or indirectly through a home server computer which then distributes it using devices like digital media players or HomeTheater PCs. The Internet can also deliver radio or television, especially through multicast, which allows a single transmission to reach multiple receivers. In the past, broadcast has been defined as either terrestrial or geographic, such as national or regional broadcast. However, with the rapid increase of the Internet, broadcast is not limited by geography, and content can reach almost any country in the world.

[0112] A storage service provider (SSP) 912 provides computer storage space and related management services. The SSP also provides periodic backups and archives. By providing storage as a service, users can order more storage as needed. Another major advantage is that the SSP includes a backup service, and if the user's computer's hard drive fails, the user will not lose all of their data. In addition, multiple SSPs can have full or partial copies of a user's data, allowing the user to efficiently access the data regardless of the user's location or the device used to access the data. For example, when the user is on the go, the user can access personal files in the home computer as well as the mobile phone.

[0113] A communication provider 914 provides connectivity to users. One type of communication provider is an Internet service provider (ISP) that provides access to the Internet. ISPs connect their customers using data transmission technologies suitable for transmitting Internet protocol datagrams, such as dial-up, DSL, cable modems, fiber, wireless, or dedicated high-speed interconnections. Communication providers can also provide messaging services, such as email, instant messaging, and SMS texting. Another type of communication provider is a network service provider (NSP) that sells bandwidth or network access by providing access to the direct backbone of the Internet. Network service providers can be composed of telecommunications companies that provide high-speed Internet access, datacom companies, wireless communication providers, Internet service providers, cable television operators, and the like.

[0114] A data exchange 904 interconnects several modules within ISP 902 and connects the modules to users 900 via network 950. Data exchange 904 can cover a small area where all modules of ISP 902 are in close proximity, or can cover a large geographic area when different modules are geographically dispersed. For example, data exchange 904 can include fast Gigabit Ethernet (or faster) within a rack of a data center, or an intercontinental virtual local area network (VLAN).

[0115] Users 900 access remote services through client devices 984 (i.e., client devices 100 in Figure 1 , which include at least a CPU, memory, a display, and I / O. Client devices can be PCs, mobile phones, netbooks, tablets, gaming systems, PDAs, and the like. In one embodiment, ISP 902 recognizes the type of device used by the client and adjusts the communication method employed. In other cases, the client device uses a standard communication method, such as html, to access ISP 902.

[0116] Figure 10 The components of an example game server device 302 that can be used to implement aspects of the various embodiments of the present disclosure are illustrated. For example, Figure 10FIG. illustrates an exemplary server system having hardware components suitable for training an AI model capable of performing various functions related to a video game and / or gameplay of the video game, in accordance with one embodiment of the present disclosure. The block diagram of the server system includes a server device 302, which can include or be a personal computer, a server computer, a game console, a mobile device, or other digital device, each of which is suitable for practicing embodiments of the present invention. Another option is that the functionality of the server device 302 can be implemented in a physical server or on a virtual machine or container server. The server device 302 includes a central processing unit (CPU) 1002 for running software applications and, optionally, an operating system. The CPU 1002 can be made up of one or more homogeneous or heterogeneous processing cores.

[0117] According to various embodiments, the CPU 1002 is one or more general-purpose microprocessors having one or more processing cores. Other embodiments can be implemented using one or more CPUs having a microprocessor architecture that is particularly suited to highly parallel and compute-intensive applications, such as media and interactive entertainment applications, configured for deep learning, content classification, and user classification among other applications. For example, the CPU 1002 can be configured to include a machine learning algorithm 410 (also referred to herein as an AI engine or deep learning engine) configured to support and / or perform learning operations related to providing various functions (e.g., predictions, suggestions) related to a video game and / or gameplay of the video game. The machine learning algorithm 410 can include a classifier 410a configured to use inputs and interactions provided during gameplay of a video game to establish and / or train an AI model (i.e., recommendation model) 410b. The AI model 410b is configured to provide suggestions to improve a measure of engagement of a group of spectators of the video game and / or gameplay of the video game. Further, the CPU 1002 includes an analyzer 1040 configured to analyze inputs and interactions and provide analysis results for generating and training the AI model 410b. The trained AI model 410b provides output in response to a particular set of player inputs, spectator interactions, where the output is according to predefined functions of the trained AI model 410b. The trained AI model 410b can be used to determine optimal suggestions to players and / or game logic to improve a measure of engagement of spectators to meet engagement criteria defined by the video game. The analyzer 1040 is configured to perform various functions related to a video game and / or gameplay of the video game, including analyzing output from the trained AI model 126b for a given input (e.g., controller input, game state data, success criteria) and providing suggestions.

[0118] Memory 1004 stores applications and data for use by CPU 1002. Storage 1006 provides non-volatile storage for applications and data and other computer-readable media, and can include fixed or removable magnetic disk drives, magnetic tape drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD, or other optical storage devices and signal transmission and storage media. User input devices 1008 communicate player input and spectator interaction from one or more players, spectators to server device 302. Examples of user input devices 1008 can include a keyboard, mouse, gamepad, touchpad, touchscreen, still camera or video recorder / camera, game controller, and / or microphone. Network interface 1010 allows server device 302 to communicate with other computer systems via an electronic communications network and can include wired and / or wireless communication over local- and wide-area networks, such as the Internet. Audio processor 1012 is adapted to generate analog or digital audio output from instructions and / or data provided by CPU 1002, memory 1004, and / or storage 1006. The components of server device 302, including CPU 1002, memory 1004, data storage 1006, user input devices 1008, network interface 1010, and audio processor 1012, are connected via one or more data buses 1022.

[0119] Graphics subsystem 1013 is also connected to data bus 1022 and to the other components of server device 301. Graphics subsystem 1013 includes graphics processing unit (GPU) 1016 and graphics memory 1018. Graphics memory 1018 includes a display memory (e.g., a frame buffer) for each pixel of an output image. Graphics memory 1018 can be integrated in the same device as GPU 1016, connected as a separate device with GPU 1016, and / or implemented within memory 1004. Pixel data can be provided directly to graphics memory 1018 from CPU 1002. Alternatively, CPU 1002 provides GPU 1016 with data and / or instructions defining the desired output image(s), and GPU 1016 generates pixel data for the output image(s) in accordance with the data and / or instructions. The data and / or instructions defining the desired output image(s) can be stored in memory 1004 and / or graphics memory 1018. In embodiments, GPU 1016 includes 3D rendering capabilities that generate pixel data for output images in accordance with instructions and data defining geometry, lighting, shading, textures, motions, and / or camera parameters for a scene. GPU 1016 can also include one or more programmable execution units capable of executing shader programs. In one embodiment, GPU 1016 can be implemented within an AI engine to provide additional processing capabilities, such as AI or deep learning functionality.

[0120] The graphics subsystem 1013 periodically outputs pixel data of an image from graphics memory 1018 for display on the display device 1010 or for projection by a projection system (not shown). The display device 1010 can be any device capable of displaying visual information in response to a signal from the server device 301, including CRT, LCD, plasma, and OLED displays. For example, the server device 301 can provide an analog or digital signal to the display device 1010.

[0121] Embodiments of the disclosure can be practiced with various computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers and the like. The disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wire- or wireless network.

[0122] In some embodiments, communication can be facilitated using wireless technology. Such technology can include, for example, 5G wireless communication technology. 5G is the fifth generation of cellular network technology. A 5G network is a digital cellular network in which service areas are divided into small geographic areas called cells, which are served by local antenna arrays and low-power automatic transceivers (transmitters and receivers). All 5G wireless devices in a cell communicate with the local antenna array and low-power automatic transceivers in the cell through radio waves via a frequency channel allocated from a pool of frequencies reused in other cells. The local antenna is connected with the telephone network and the Internet through a high-bandwidth optical fiber or wireless backhaul. As in other cell networks, a mobile device moving from one cell to another is automatically transferred to the new cell. It should be understood that 5G networks are merely an exemplary type of communication network, and embodiments of the disclosure can utilize earlier generations of wireless communication or wired communication and newer generations of wired technology or wireless technology after 5G.

[0123] With the above embodiments in mind, it should be understood that the disclosure can employ various computer-implemented operations involving data stored in computer systems. These operations are those requiring physical manipulation of physical quantities. Any of the operations described herein that form part of the disclosure are useful machine operations. The disclosure also relates to a device or an apparatus for performing these operations. The apparatus can be specially constructed for the required purposes, or it can be a general purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general purpose machines can be used with computer programs written in accordance with the teachings herein, or it can be more convenient to construct a more specialized apparatus to perform the required operations.

[0124] The present disclosure can also be embodied as computer readable code on a computer readable medium. Another option is to download the computer readable code from a server with the data exchange interconnection described above. The computer readable medium is any data storage device that can store data which can thereafter be read by a computer system. Examples of computer readable media include hard drives, network attached storage devices (NAS), read-only memory, random-access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. The computer readable medium can include computer readable tangible medium distributed over a network-coupled computer system so that the computer readable code is stored and executed distributed over a computer system in a distributed fashion.

[0125] Although the method operations were described in a specific order, it should be understood that other housekeeping operations can be performed in between operations, or operations can be adjusted so that they occur at slightly different times, or can be distributed in different order, or distributed across different systems, etc. without departing from the scope of the present disclosure.

[0126] Although the foregoing disclosure has been described in detail for purposes of clarity, it will be apparent that certain changes and modifications can be made without departing from the scope of the appended claims. Accordingly, the embodiments are to be considered illustrative and not restrictive, and the disclosure is not to be limited to the details given herein, but can be modified within the scope and equivalents of the embodiments.

[0127] It should be understood that various features of the herein disclosed embodiments can be combined or assembled into particular embodiments in any manner that is within the scope of the embodiments defined herein. Thus, the examples provided are only some of the possible examples and not all possible examples can be provided by combining various elements of the embodiments. In some examples, some embodiments can include fewer elements than the examples provided, and this does not depart from the spirit of the disclosure or equivalent embodiments.

Claims

1. A method comprising: Requests are received from passive viewers for watching a video game running on a game cloud server, wherein the video game is a multiplayer video game that generates game execution data using input from multiple players, and the game execution data is used to generate a player camera view for each of the multiple players and an active viewer camera view for each active viewer watching the game execution of the video game. Analyze the user profiles of the passive viewers to determine their viewing preferences; as well as Selected camera views from the player's camera view and the active viewer's camera view are identified and forwarded to a client device associated with the passive viewer for rendering specific camera views from the player's camera view and the active viewer's camera view, based on the passive viewer's user profile and corresponding to actions currently occurring in the video game during gameplay.

2. The method according to claim 1, wherein, The dynamic switching is related to changes detected in the actions occurring in the video game, and is forwarded to a specific camera view in the player's camera view or a specific camera view in the active viewer's camera view of the client device of the passive viewer to capture the changes in the actions.

3. The method according to claim 1, wherein, When no change is detected in the currently occurring action within the video game during a predefined time period, the dynamic switch is performed. The dynamic switch is related to the expiration of the predefined time period and provides different perspectives of the currently occurring action in the video game.

4. The method according to claim 1, wherein, The dynamic switching occurs essentially in real time.

5. The method according to claim 1, wherein, The passive viewers' viewing preferences are collected over time based on the content they watch, and the passive viewers' user profiles are dynamically updated.

6. The method according to claim 5, wherein, The viewing preferences of the passive viewers are dynamically updated in the user profile based on the preference input received from the passive viewers.

7. The method according to claim 1, wherein, The viewing preferences of the passive viewers are dynamically updated in the user profile based on the preference input received from the passive viewers.

8. The method according to claim 1, wherein, During gameplay, the camera view of each of the multiple players and the camera view of each active viewer are dynamically updated to capture views of the changes in the actions occurring in the video game.

9. The method according to claim 1, wherein, The actions that occur in the video game are based on activities performed by players among the plurality of players, and the player camera view and the active viewer camera view capture different perspectives of the actions in relation to the activities.

10. The method according to claim 1, wherein, The dynamic switching continues during the gameplay of the video game and enables the generation of a recommendation stream specific to the passive audience. The recommended stream includes one or more player camera views and / or one or more active viewer camera views, wherein each player camera view and each active viewer camera view includes a dissimilar view of the action occurring in the video game.

11. The method according to claim 1, wherein, Multiple actions occur during gameplay of the video game, each of which is associated with one or more player camera views and one or more active viewer camera views. The one or more player camera views and the one or more active viewer camera views associated with each action capture dissimilar views of the corresponding action occurring in the game scene. The one or more player camera views and the one or more active viewer camera views generated for each action include sufficient data to construct a three-dimensional representation of the game scene of the video game associated with the action.

12. The method according to claim 1, wherein, Each player's camera view captures a view of the actions occurring in the video game from the perspective of one of the multiple players, and Each active viewer camera view captures a view of the action that is different from the view of the action captured in the player camera view of each of the plurality of players, and the active viewer camera view generated for the active viewer is based on the background of the action and the input provided by the active viewer.

13. The method according to claim 12, wherein, The actions that occur in the video game are in response to game input provided by the player.

14. The method according to claim 1, wherein, The dynamic switching includes presenting either the player's camera view or the active viewer's camera view selected by the passive viewer during the duration of the action occurring in the video game.

15. The method according to claim 1, wherein, The game progress data is aggregated game progress data, which includes game progress data generated from the game progress of multiple players currently playing the video game. This aggregated game progress data is used to generate player camera views for the multiple players and active viewer camera views for multiple active viewers for each action occurring in the video game. Each player camera view captures a view of the video game from the perspective of a specific player among the plurality of players, using that specific player's game input, and each active viewer camera view captures a view of the video game based on view specifications provided in the input of a specific active viewer among the plurality of active viewers. The dynamic switching includes (a) switching only between the player camera views, or (b) switching only between the active viewer camera views, or (c) switching between the player camera views and the active viewer camera views to provide views of game scenes associated with different actions occurring in the video game.

16. The method according to claim 1, wherein, Each player camera view in the featured camera view of the player camera view and each spectator camera view in the featured camera view of the spectator camera view are different, and capture the view of the action currently occurring in the video game during gameplay.

17. The method according to claim 1, wherein, Forwarding selected camera views in the player's camera view and the active viewer's camera view includes dynamically switching between specific camera views in the selected camera view of the player's camera view and specific camera views in the selected camera view of the active viewer's camera view based on the background of the action currently occurring during the gameplay of the video game.

18. The method according to claim 1, wherein, The dynamic switching occurs without input from the passive audience, and the operation of the method is performed by the server of the game cloud server.

Images