Camera view selection processor for passive audience viewing

Abstract

A method and system for enhancing a spectator's viewing experience includes receiving a viewing request for gameplay of a video game executed on a game cloud server. In response to the request, the spectator is initially provided with one of a generated player camera view or an active spectator camera view of the gameplay. The system dynamically switches between the player camera view and the active spectator camera view based on the context of action occurring in the player camera view and the active spectator camera view. The dynamic switching correlates with the time of action occurring in a game scene of the video game.

Description

Technical Field

[0001] The present disclosure relates to providing a recommended stream of video game gameplay to viewers, and more particularly, to providing a recommended stream by different camera views of game scenes that capture different aspects of the gameplay.

Background Art

[0002] In recent years, the popularity of video games has been increasing. There are two types of video games. The first type of video game is executed on a client device (such as a mobile device, a PC, a laptop computer, etc.), and the client is connected to a server so that metadata related to the gameplay can be shared with the server. The other type of video game is a streaming video game, in which the video game is executed on one or more servers that are part of a game cloud, and the game data is streamed to a client device and rendered. Streaming video games, especially massively multiplayer online (MMO) games, are more popular because a huge number of users (such as players, viewers) can access them simultaneously via a network of computers distributed over a wide area.

[0003] In the video game industry, the concept of cloud gaming is on the rise. Cloud gaming allows video games to run on one or more cloud servers using resources available in a game cloud system, enabling multiple players and viewers to access the video game from anywhere with a network connection. Massively Multiplayer Online (MMO) games are an example of video games that utilize cloud gaming, and such video games are becoming increasingly popular. Current trends include the inclusion of highly advanced graphics in video games (e.g., MMO games), which necessitates a large amount of computing resources, and cloud gaming provides the necessary computing resources in a centralized manner to meet the demands of modern game engines. Video games run on one or more servers in the game cloud system and stream gameplay data to the user's client device. An advantage of running video games on a game cloud system is that, because the resources required to run the game are centralized, the user's client device does not need to own specific hardware to run the video game. The client device only needs to have enough hardware to receive, decode, and render gameplay while enjoying a high-quality video game experience.

[0004] One or more players play a video game, and multiple spectators can access to watch the gameplay of one or more players. Player input provided during gameplay is used to update the game state of the video game, generating gameplay data that is streamed to the client devices of one or more players and spectators. This gameplay data is used to render the game scenes of the video game on each of the player's and spectator's client devices. As long as cloud gaming remains popular, the sophistication of graphics, and therefore the quality of video games configured for cloud gaming, will continue to improve.

[0005] The embodiments of this disclosure were made against this background. [Overview of the Initiative]

[0006] Embodiments of this disclosure relate to a system and method for recommending a stream of video game gameplay to an audience interested in video game gameplay. The video game may be a single-user video game or a massively multiplayer online (MMO) video game, which may run on and be accessed on a cloud gaming system for gameplay by multiple players who are in the same location or in different locations (i.e., distributed across different geolocations). Game inputs provided by the players are used to influence the game outcome and generate gameplay data. The gameplay data is processed to generate frames of game scenes, which are streamed to the players' client devices and rendered. The frames of game scenes provided to each player provide a camera view of the video game's game scene based on the viewpoint of each player (i.e., a "player camera view"). Multiple audiences may be interested in watching video game gameplay. Some of the multiple audiences may choose to actively participate in watching video game gameplay by selecting a specific camera view of the gameplay they want to see, while other audiences may simply choose which video game they want to see. For viewers who actively participate in the selection of a particular camera view (referred to herein as “active viewers”), the system may generate a specific camera view (i.e., an active viewer camera view) and transfer the generated view to the viewer who requested that view. For viewers who simply select a video game they wish to watch without actively specifying any particular camera view, the system determines which camera view (i.e., player camera view or active viewer camera view) to offer to these viewers. Viewers who simply choose to watch a video game without actively selecting any particular camera view are also referred to herein as “passive viewers”. In the case of passive viewers, the system switches between different camera views generated for the video game so that passive viewers are always presented with either the player's player camera view or the active viewer camera view for active viewers.The switching between the two camera views is based on the context of the actions occurring in each camera view. During a video game, passive viewers are presented with multiple camera views (either the player camera view or the active viewer camera view), each presenting a view of the actions occurring in the video game. The multiple camera views presented to passive viewers constitute a recommendation stream. The camera views included in the recommendation stream are a subset of camera views captured during gameplay, providing passive viewers with details of the video game that are of interest to them.

[0007] In one embodiment, a method is provided. The method includes running a video game on a game cloud server in response to a play request for gameplay of a video game received from a player. Running the video game generates gameplay data, which is processed to generate a player camera view for provision to the player during gameplay and an active audience camera view for provision to active audiences who access the video game and watch the player's gameplay. A request to view gameplay of the video game is received from a passive audience. In response to the request, either the player camera view or the active audience camera view is presented to the passive audience. Presentation may include dynamically switching between the player camera view and the active audience camera view. Dynamic switching is based on the context of actions occurring in the player camera view and the active audience camera view. Dynamic switching is performed without input from the passive audience.

[0008] In one embodiment, the player camera view and the active spectator camera view are dynamically updated during gameplay to capture a view of the action occurring in the video game.

[0009] In one embodiment, the player camera view or active spectator camera view selected for dynamic switching is associated with the timing of actions occurring in the video game.

[0010] In one embodiment, dynamic switching to the player camera view or the active spectator camera view is further based on the passive spectator profile.

[0011] In one embodiment, the passive audience profile identifies the passive audience's viewing preferences, which are collected over time based on the content the passive audience has viewed.

[0012] In one embodiment, the viewing preferences of passive audiences are dynamically adjusted in a profile based on preference input received from the passive audiences.

[0013] In one embodiment, actions occurring in a video game are based on activities performed by the player, and the player camera view and the active spectator camera view capture actions related to those activities.

[0014] In one embodiment, a recommendation stream is generated for a passive audience through continuous dynamic switching during video game gameplay. The recommendation stream includes a combination of one or more player camera views and / or one or more active audience camera views. Each player camera view and each active audience camera view is dynamically generated and includes distinct actions occurring in the video game.

[0015] In one embodiment, multiple actions occur during gameplay of a video game, and each of these actions is associated with one or more player camera views and one or more active spectator camera views. The one or more player camera views and one or more active spectator camera views associated with each action capture a separate view of the game scene in the video game where the corresponding action is occurring. The one or more player camera views and one or more active spectator camera views generated for an action contain sufficient data to construct a three-dimensional representation of the game scene in the video game associated with the action.

[0016] In one embodiment, processing gameplay data includes replaying the video game to generate an active audience camera view that captures the actions occurring in the video game at a desired angle specified by the active audience input. The replay is performed using the gameplay data available at the time input from the active audience is received.

[0017] In one embodiment, the player camera view captures a view of the actions occurring in the video game from the player's point of view. The active spectator camera view is generated based on input provided by the active spectator. The active spectator camera view captures a view of the actions that is different from the view of the actions captured by the player camera view.

[0018] In one embodiment, dynamic switching includes presenting a player camera view or an active audience camera view to a passive audience for the duration of an action occurring in a video game.

[0019] In one embodiment, gameplay data is an aggregation of gameplay data from multiple players currently playing a video game. Using the aggregated gameplay data, player camera views for multiple players and active spectator camera views for active spectators are generated for each action that occurs in the video game. Each player camera view captures a view of the video game from the viewpoint of one of the multiple players using the game input of that player, and each active spectator camera view captures a view of the video game according to the view specifications provided by a specific active spectator input. Dynamic switching includes presenting only player camera views, only active spectator camera views, or a combination of player camera views and active spectator camera views, which capture views of game scenes associated with different actions that occur in the video game.

[0020] In one embodiment, dynamic switching is further based on the profile of the passive audience.

[0021] In one embodiment, dynamic switching is performed by generating a recommendation model using machine learning logic. The recommendation model is generated and trained using video game gameplay data. The gameplay data is generated in response to player game inputs and active audience inputs. From the recommendation model, outputs that correlate with the context of actions occurring in the video game are identified. These outputs are used to identify the player camera view or the active audience camera view for dynamic switching.

[0022] In another embodiment, a method is disclosed. The method includes detecting a viewing request for a video game running on a game cloud server. The request is received from a passive viewer. Multiple camera views generated during gameplay of the video game are identified. The multiple camera views include a player camera view generated for each player and an active viewer camera view generated for each active viewer following the gameplay of the video game, each camera view of the multiple camera views capturing a separate view of the game scene of the video game associated with a corresponding action occurring in the video game. The method further includes dynamically switching between player camera views and active viewer camera views selected from among the multiple camera views associated with each action to be returned to the passive viewer upon request. The dynamic switching is performed without input from the passive viewer and is based on the context of the action occurring in each player camera view and active viewer camera view selected for that action.

[0023] Other aspects and advantages of this disclosure will become apparent from the following embodiments for carrying out the invention, which illustrate the principles of this disclosure as examples, in conjunction with the accompanying drawings.

[0024] This disclosure can be best understood by referring to the following description in conjunction with the attached drawings. [Brief explanation of the drawing]

[0025] [Figure 1] A game cloud system of a simplified concept is shown that collects game inputs from players at different geolocations and adjusts the output of a video game for use in providing a recommendation stream to viewers, according to one embodiment of the present disclosure. [Figure 2A] A schematic diagram of different camera views captured in a video game play used for providing a recommendation stream is shown, according to one embodiment of the present disclosure. [Figure 2B] An enlarged view depicting different camera views captured for different users, namely players, active viewers, is shown, according to one embodiment of the present disclosure. [Figure 3A] A simplified block diagram of a game cloud system is shown that collects and processes game inputs from multiple players and inputs from multiple active viewers to generate corresponding player camera views and active viewer camera views for each of the multiple players and active viewers, according to one embodiment of the present disclosure. [Figure 3B] A simplified block diagram of a game cloud system used to provide different camera views to passive viewers is shown, according to one embodiment of the present disclosure. [Figure 4] A simplified conceptual block diagram of a view recommendation engine used to generate different camera views that make up a recommendation stream provided to passive viewers is shown, according to one embodiment of the present disclosure. [Figure 5] A simplified block diagram of various modules within a view recommendation engine used to generate a recommendation stream for viewers is shown, according to one embodiment of the present disclosure. [Figure 6] A simplified block diagram of various modules within a feature processing engine used to extract and evaluate various features of game play data provided to a recommendation engine that identifies different camera views for presentation to passive viewers is shown, according to one embodiment of the present disclosure. [Figure 7-1]A-1 to C-1 show examples of switching camera views presented to different passive viewers watching gameplay of a video game, according to one embodiment of the present disclosure. A-2 to C-2 show example recommendation streams generated from the switching camera views identified in A-1 to C-1, provided to different passive viewers by a view recommendation engine, according to one embodiment of the present disclosure. [Figure 7-2] D-1 to E-1 show examples of switching camera views presented to different passive viewers watching gameplay of a video game, according to one embodiment of the present disclosure. D-2 to E-2 show example recommendation streams generated from the switching camera views identified in D-1 to E-1, provided to different passive viewers by a view recommendation engine, according to one embodiment of the present disclosure. [Figure 8] This diagram shows various operations of a method used in generating a recommendation stream to return to the audience, according to one embodiment of the present disclosure. [Figure 9] This disclosure provides an exemplary embodiment of an information service provider architecture according to one embodiment of this disclosure. [Figure 10] A simplified block diagram of an exemplary cloud game server used to run an instance of a video game according to one embodiment of the present disclosure is shown. [Modes for carrying out the invention]

[0026] The following description includes many specific details to provide a complete understanding of this disclosure. However, it will be apparent to those skilled in the art that this disclosure can be implemented without some or all of these specific details. In other examples, well-known process steps are not described in detail to avoid obscuring this disclosure.

[0027] Generally, a player selects a video game (simply referred to as "the game") for gameplay and provides game input. Game logic processes the game input and generates gameplay data. This gameplay data is used to generate various camera views for each scene of the video game that the player accesses. The streaming capabilities of video games enable live streaming of gameplay over networks such as the internet. Spectators can access the video game and watch the player's gameplay via websites, game platforms, mobile applications, content provider platforms, or any other application / content platform. Audiences may wish to follow the gameplay of a particular player or the overall gameplay of the game. Advanced video game design allows for the generation of different camera views for each action occurring in each game scene accessed by the player. For example, gameplay data from a video game can be used to generate a camera view for each player playing the video game (i.e., a player camera view). The camera view generated for each player can capture a view of the game scene where a particular action is occurring from the player's perspective. In one embodiment, a particular action may occur in response to a player's game input. In an alternative embodiment, a particular action may occur in response to game input from another player(s) or in accordance with the natural progression of events in the video game. When an audience wishes to follow the gameplay of a game, the video game may generate an audience camera view in addition to the player camera view for each audience member accessing the video game to watch the gameplay. The audience member can provide input specifying the camera view they wish to view for each action occurring in each game scene of the video game, and the audience camera view is generated according to the details provided by the audience member in the input. The view of the game scene captured by each audience camera view generated for each action in the video game will differ from the view captured by the player camera view generated for the same action. An audience member who actively provides input specifying the camera view they wish to view during gameplay is referred to herein as an “active audience member.” Therefore, the audience camera view generated for an active audience member will hereafter be referred to as an “active audience camera view” (ASCV). Alternatively, the audience may wish to follow the overall gameplay of the game. In this case, the audience does not actively provide input specifying the camera view they want to see. Instead, the audience relies on the system to provide camera views of the different actions that occur in the video game. Because the audience does not actively provide input to select a camera view, they are referred to herein as “passive audience.” Using game logic, for each action, either a player camera view or an active audience camera view is selected to be presented to the passive audience. Given the large number of players playing MMO video games and the large number of audiences watching video games, video games generate a multitude of camera views that capture views of the different actions that occur in the video game during gameplay.

[0028] Generally, when a passive audience wants to watch a video game's gameplay, they are presented with all the camera views generated for each action that occurs in the video game. The passive audience must manually review each camera view for each action in order to select a specific camera view for the action they want to watch. Given the enormous number of camera views generated for each action, this can be a daunting task. Having to manually review various camera views is time-consuming and laborious, leading passive audiences to become frustrated with the game's gameplay and lose interest. Since audience interest often translates to higher revenue from game purchases or sponsorships, losing audience interest can lead to lost revenue for the game developer.

[0029] Various embodiments of this disclosure provide systems and methods to address such problems, wherein gameplay data of a game is collected from multiple players in a gameplay session (e.g., the current or a previous session), various camera views available for each action in a game scene are identified, and specific camera views that provide a passive viewer with a desired view of the action in the game scene are intelligently identified. Specific camera views for each action are identified based on the passive viewer's profile, which identifies the passive viewer's viewing preferences based on the passive viewer's viewing history collected over time. In some cases, passive audience viewing preferences can be dynamically adjusted based on preference inputs provided by the passive audience. A recommendation stream is generated using various camera views of the video game provided to the passive audience while they are watching the video game. The camera views presented to the passive audience and included in the recommendation stream may include only player camera views selected from the gameplay of a single player or multiple players, only active audience camera views generated for different active audiences, or a combination of both player and active audience camera views. As mentioned above, the player's player camera view captures the game scene in which action is taking place from the player's perspective, while the active audience camera view either extends beyond the range of the game scene captured by the player camera view or captures a different game scene from the one captured by the player camera view. Each camera view included in the recommended stream is based on the passive audience's profile and is selected to match one or more of the passive audience's viewing preferences. By selecting camera views based on the passive audience's profile, passive audiences can view gameplay according to their viewing preferences, and the camera view selection is made without requiring input from the passive audience, thereby providing them with a richer viewing experience.

[0030] A recommendation model is generated using a view recommendation engine with machine learning algorithms, based on game input from the player and gameplay data generated during video game gameplay. The recommendation model is continuously trained with additional game input received from the player and additional gameplay data generated by the video game in response to that additional game input. The recommendation model identifies outputs that match the viewing purpose defined by the viewing preferences included in the passive viewer's profile. Using the outputs from the recommendation model, specific camera views to present to the passive viewer are identified, including views of the game scene in which the action is taking place. Because the specific camera views identified from the output of the recommendation model are selected according to the passive viewer's viewing preferences, they are more relevant to the passive viewer than the views provided by the video game.

[0031] With an overview of the embodiments of the present invention, further illustrative details of various embodiments will be described with reference to various drawings.

[0032] Figure 1 provides an overview of a game cloud system (GCS) 10 used to access and play games. The GCS 10 includes multiple client devices 100 (100-1, 100-2, 100-3, ..., 100-n) distributed across a single geolocation or different geolocations and connected communicably to a game cloud site 300 via a network 200. The GCS 10 is configured to host multiple games and other interactive applications such as social media applications and content provider applications. The GCS 10 can be accessed from a single geolocation or from multiple geolocations. The client devices 100 may be any type of client computing device having a processor, memory, and communication capabilities to access a network 200 such as LAN, wired, wireless, or 4G / 5G, and may be portable or non-portable. The client devices 100 may run an operating system and include a network interface for accessing the network 200, or they may be thin clients having a network interface for communicating with the game cloud site 300 via the network 200, where the game cloud site 300 provides computing capabilities. For example, the client device may be a smartphone, mobile device, tablet computer, desktop computer, personal computer, wearable device, internet-connected television, or other digital device having a portable form factor, including a hybrid or monitor or touchscreen.

[0033] A client device 100 with 5G communication capabilities may include a mobile device or any other computing device capable of connecting to a 5G network. In one embodiment, the 5G network is a digital cellular network in which the service area is divided into multiple “cells” (i.e., small geographical areas). Analog data generated by a mobile device is digitized and transmitted as radio waves to local antennas within the cell using frequency channels, which can then be reused in geographically distant cells. The local antennas are connected to the Internet and telephone networks by high-bandwidth optical fiber or other similar radio connections. Because the 5G network uses high-frequency radio waves for communication, it can transmit data at higher data rates, resulting in lower network latency.

[0034] Players can access video games available on GCS10 using their user accounts. In response to a player's request for access to a game for gameplay, the player's user account is verified based on user account 304, which is held in user data store 305. Before providing access to a video game, the request is also verified based on game data store 306 to determine whether the player is qualified to access and play the video game. This verification is performed by identifying all game titles available on game cloud site 300 that the player is qualified to view or play, and then activating the game titles included in the player's request based on the identified game titles. Game data store 306 maintains a list of game titles that are hosted or may be hosted on GCS10, and when a new game is introduced, the game title, game code, and information related to the new game are updated in game data store 306. While various embodiments are described in relation to video games (also referred to as "games"), it should be noted that embodiments can be extended to include any other interactive applications.

[0035] After successful verification of the user and request, the game cloud site 300 identifies a data center where the game may be hosted in order to load the game associated with the game title identified in the request, and sends a signal to the identified data center. In some embodiments, multiple data centers may host or be capable of hosting the game. In these embodiments, the game cloud site 300 identifies a data center geographically close to the player's geolocation. The player's geolocation may be determined using, to name a few examples, a Global Positioning System (GPS) mechanism within the client device 100, the client's IP address, the client's ping information, social interactions and other interactions conducted through the client device 100. Of course, it should be noted that the above-described methods for detecting the player's geolocation are provided as examples, and the player's geolocation may be determined using other types of mechanisms or tools. Identifying a data center close to the player's geolocation may reduce latency when transmitting game-related data between the player's client device 100 and the game running in the identified data center 301. The data center 301 may include multiple game servers 302, and a game server 302 is selected based on the resources available to the game server 302 for hosting the game. In some embodiments, an instance of the game may run on one or more game servers 302 within the identified data center 301.

[0036] In some embodiments, the identified data center 301 may not have the resources (e.g., bandwidth, processing power, etc.) necessary to host the game. In such embodiments, the game cloud site 300 may identify a second data center that is geographically close to the player's geolocation and has the resources necessary to host the game.

[0037] The game cloud site 300 loads the game onto one or more game servers 302 within an identified data center 301. The one or more game servers 302 contain the hardware / software resources necessary to meet the game's requirements. The game servers 302 may be any type of server computing device available at the game cloud site 300, including standalone servers. Furthermore, the game servers 302 may manage one or more virtual machines on a host that support game processors running instances of the player's game.

[0038] In some embodiments, one or more servers 302 may include multiple game consoles (or computing devices) 303, and a game cloud site 300 may identify one or more game consoles or computing devices 303 within one or more identified servers 302 and load games. Each of the one or more game consoles / computing devices 303 may be an independent game console or computing device, or it may be a rack-mount server or a blade server. A blade server may include multiple server blades, each having the circuitry and resources necessary to instantiate, for example, a single instance of the game. Naturally, the aforementioned game consoles are illustrative and should not be considered limiting. Other types of game consoles or computing devices, including other forms of blade servers, may also be used to run an identified instance of the game. Once one or more game consoles or computing devices are identified, the game's generic game-related code is loaded onto one or more game consoles / computing devices and made available to the player.

[0039] In other embodiments, the video game may run locally on the client device 100, and metadata from the running video game may be transmitted via the network 200 to a game cloud server(s) (hereinafter simply referred to as "game server") 302 in an identified data center 301 of the game cloud site 300, thereby influencing the game state and sharing gameplay data with other players and spectators.

[0040] Game inputs that affect the game state of the game may be provided from input devices such as a mouse 112 or keyboard (not shown) associated with the client device 100, or from a control interface (e.g., a touchscreen), or from a controller 120 that is communicatively connected to the client device 100. A recommendation model (i.e., an artificial intelligence (AI) model) is created using gameplay data generated from the player's game inputs during gameplay of the video game. The recommendation model is further trained using additional gameplay data generated from ongoing game input provided by the player during a game session. The trained recommendation model is used to identify the camera views provided to each player. These camera views capture a view of a game scene within the video game where one or more actions are occurring during gameplay, and are designed to capture the game scene from each player's perspective. Thus, the camera views provided to each player are also referred to herein as “player camera views.” The trained recommendation model is also used to identify camera views provided to different active viewers, and the identified camera views satisfy each viewer’s viewing objectives. Active viewers are those who access the video game to watch gameplay and actively provide input to identify different views of game scenes associated with each action in the video game they wish to watch. Based on the input from active viewers, the video game is replayed to generate specific camera views for the active viewers. The camera view generated for the active audience, hereafter also referred to as the "active audience camera view," captures the view of the game scene and captures the action from a different viewing angle, viewing depth, or viewing direction than that captured by the camera view provided to each player.

[0041] In some cases, non-active viewers may access and watch video games. In this case, the viewer selects the video game to watch, but provides no other input to select a particular camera view. After selecting the video game, the viewer simply sits and watches the camera views that capture different stages of the video game's gameplay. In this case, the viewer does not provide any input that influences the generation of the camera views, and instead relies on the system to provide the viewer with camera views that capture different actions within the video game; therefore, in this specification, the viewer will be referred to as a "passive viewer." A view recommendation engine running on the game cloud site 300 identifies specific camera views and provides them to passive viewers. The view recommendation engine uses a recommendation model to identify specific camera views from among the camera views in a video game and return them to passive viewers. Camera views capturing different actions within the video game are identified according to the passive viewer's profile. Each camera view identified for passive viewers captures a view of the game scene in which action is taking place, is presented for the duration of the action, and then dynamically switches to another camera view that captures the next action occurring in the video game. Thus, while watching video game gameplay, passive viewers may be presented with one or more player camera views and / or one or more active viewer camera views.

[0042] The games running on the game cloud site 300 may be single-player games or multiplayer games. In some embodiments, the video game may be a massively multiplayer online (MMO) game, which allows multiple players across different geolocations to access and play the video game. As a result, the player camera view presented to the passive audience may include the player camera view of a single player or multiple players. Similarly, the active audience camera view presented to the passive audience may include the active audience camera view of a single active audience or multiple active audiences. The different camera views provided to passive viewers are based on viewing preferences identified from the passive viewer's user profile. Therefore, the camera views allow passive viewers to view different actions according to their viewing preferences, thereby providing them with a richer game viewing experience. Frames from specific camera views identified for passive viewers are transferred to the game engine, processed, encoded, and streamed to the passive viewer's client device, where the stream is decoded and rendered.

[0043] A game engine (not shown) may include a multiplayer distributed game engine that is communicatively connected to the game logic of a game. Generally speaking, a game engine is a software layer that acts as the foundation for games such as MMO games and provides a framework used in the development of video games. The game engine extracts the details of performing common, related tasks required for any game (i.e., game engine tasks), while the game developer provides the game logic that provides the details of how the game is played. The game engine framework includes multiple reusable components for handling several functional parts of the game (i.e., core functions) that bring the video game to life. Basic core functions handled by the game engine may include physical properties (e.g., collision detection, collision response, trajectory, movement of objects based on gravity, friction, etc.), graphics, audio, artificial intelligence, scripting, animation, networking, streaming, optimization, memory management, threading, localization support, and much more. Reusable components include a process engine used to handle core functions identified in relation to the game.

[0044] During gameplay, the game engine manages the game's game logic, collects input from one or more players received from one or more client devices 100, and sends it to the game logic. The game engine further manages the allocation and synchronization of its functional components in an optimal manner to process the game data generated by the game logic, and generates frames of game data that are returned to the client devices 100 for rendering. Currently, various game engines are available to provide different core functionalities, and a suitable game engine may be selected based on the specified functionalities for running the video game. In response to requests to view game scenes, recommendation streams generated by the view recommendation engine are processed and encoded by the game engine and streamed to the audience's client devices.

[0045] The game inputs provided by the player during gameplay correspond to the activities the player performs within the video game. For example, an activity performed by the player may trigger the execution of a specific action in a game scene of the video game. The player's game inputs, the activities performed, and the actions that occur in the video game are part of the telemetry data used to generate gameplay data 308. The gameplay data 308 and telemetry data are stored in the gameplay data store 307. The game inputs provided by the player are processed by the game logic and affect the game state of the video game. The game state of the video game identifies the overall state of the video game at a particular point in time and is influenced by the complexity of the effect of the player's game inputs on the gameplay of the video game. If the video game is an MMO game, inputs from multiple players are used to influence the overall game state of the video game. The gameplay data generated regarding gameplay includes each player's saved data. The player's saved data includes any game customizations provided by the player regarding the video game. Telemetry data captures, to name a few, the characteristics of each activity (such as what the player attempted, what the player achieved, what the player failed at), the player's player attributes, the player's game customizations, and game features. Player attributes can be updated in the player profile stored in the user data store 305. The gameplay data also includes image data related to game scenes accessed during gameplay. The image data is used to generate various camera views provided to the player, active viewers, and passive viewers.

[0046] Figure 2A shows various camera views that may be generated with respect to a game during gameplay in an exemplary embodiment. It is shown that multiple players access a game cloud site 300 and select game 1 of a video game for gameplay. Gameplay data is generated using the game inputs provided by each player. The gameplay data is processed to generate frames of player camera views that are transferred to each player. One or more active viewers may choose to follow the gameplay of one or more players. These active viewers provide inputs used to identify specific camera views that capture each action that occurs in the video game. Figure 2A shows players P1 to P6 currently playing Game 1. Each player may be progressing through Game 1 at a different pace. Each player is provided with their own player camera view of the game scene in which an action is taking place. Actions can occur in response to game input provided by one or more players, or based on the game logic of the video game. The player camera view (PCV) provided to each player shows the game scene from the player's perspective. Therefore, player P1 is presented with player camera view PCV1, showing the game scene of Game 1 from player P1's perspective; player P2 is presented with player camera view PCV2; player P3 is presented with player camera view PCV3, and so on. Game inputs provided by each player are sent to the game cloud site 300 via network 200. The game logic of Game 1, which runs at the game cloud site 300, uses each player's input to influence the game state of Game 1 and generate gameplay data. The gameplay data generated from each player's input is processed to produce image frames that are returned to each player as a player camera view. Each player's player camera view shows the current game state of Game 1.

[0047] One or more active spectators also sign in to the game cloud site 300 to watch the gameplay of one or more players. For example, spectators S1 and S2 may choose to watch the gameplay of player P1, spectator S3 may choose to watch the gameplay of player P3, and spectator S4 may choose to watch the gameplay of player P5. With respect to an action that occurs in the video game and generates one or more corresponding player camera views, each active spectator actively provides input during gameplay to specify the particular camera view they want to capture. The game logic uses gameplay data corresponding to the time of the action to rerun the video game and generate an active audience camera view for the active audience according to the details contained in the input. The input from the active audience can identify specific directions, depths, angles, clarity, etc., that you want to capture in the game scene associated with each action, and the active audience camera view is generated accordingly. The active audience camera view generated for each action for each active audience captures a different view of the game scene than the view captured by the corresponding player camera view(s) generated for the same action. For example, based on the input of active audience AS1, active audience camera view ASCV1.1 (i.e., active audience camera view 1 of the game scene for player 1) is generated with respect to the game scene captured by player camera view PCV1 of player 1. Similarly, based on the input from the active spectator AS2, an active spectator camera view ASCV1.2 (i.e., active spectator camera view 2 of player 1's game scene) is generated with respect to the game scene captured by player 1's PCV1. Likewise, based on the input provided by the active spectator AS3, an active spectator camera view ASCV3.3 is generated with respect to the game scene captured by player 3's PCV3, and based on the input from spectator AS4, an active spectator camera view ASCV5.4 is generated with respect to the game scene captured by player 5's PCV5, and so on. Each active spectator camera view generated from the active spectator input can be different and can also identify different angles, depths, directions, clarity, etc., of the capture of the same game scene that the player is viewing. The generated active spectator camera views can display fewer or more features of the game scene.

[0048] Figure 2B shows enlarged views of various camera views of a game scene in Game 1 where an action is taking place. The action can be obtained in response to game input from player P1 or based on the game logic of the video game. Player P1 is presented with player camera view PCV1, which captures a view of game object 101 in the game scene. Based on inputs provided by active spectators S1 and S2, two active spectator camera views are generated for this game scene. Thus, the active spectator camera view ASCV1.1, generated in response to the input of active spectator S1, shows the camera view of the game scene captured by player P1's PCV1 from different viewing angles. Therefore, ASCV1.1 includes a view of game object 101 that was included in PCV1, as well as a view of a first game object 102 that was outside the player camera view PCV1. This is because the active spectator camera view ASCV1.1 was captured from a position beyond PCV1, as shown in Figure 2B. Similarly, the active spectator camera view ASCV1.2 generated from the input of active spectator S2 captures a different perspective of the game scene captured in PCV1 and includes a view of a second game object 103 in addition to game object 101 captured in PCV1. The active spectator camera view ASCV1.2 captures game object 101 from a different angle than that captured by PCV1 and ASCV1.1. Active spectator camera views generated from different active spectator inputs may capture game object 101 from a different angle than that captured by PCV. For example, one active spectator camera view (not shown) may capture an overhead camera view of game object 101, while another ASCV may show a view from the opposite side of game object 101 as shown in PCV1. The game logic is configured to generate different views of each game scene of gameplay in which action is taking place, based on inputs from different active spectators, in order to capture different features of the gameplay.

[0049] When a passive viewer selects a video game to watch, the view recommendation engine running on the game cloud site 300 is configured to extract different camera views generated for each action that occurs in each game scene, and then select a specific camera view from the extracted camera views to present to the passive viewer regarding that action. The specific camera view is selected based on the passive viewer's viewing preferences specified in the passive viewer's user profile. Passive viewers simply select the video game they want to watch, and the view recommendation engine performs the task of examining the different camera views generated for each action and identifying the camera view best suited to the passive viewer. Each camera view presented to the passive viewer corresponds to an action occurring in the game scene. The camera view presented for each action is one of either the player camera view or the active viewer camera view generated for the action, and is presented for the duration of the action captured within the camera view.

[0050] The view recommendation engine dynamically switches between the player camera view and the active spectator camera view, with the dynamic switching based on the context of the actions occurring in each camera view. For example, the video game might be a car racing game, and the camera views capture different views of the race. Player 1's player camera view PCV1 captures the game scene of the race from Player 1's point of view, showing that Player 1's car is ahead of competitors with no cars in front. Spectator S1's active spectator camera view captures the same game scene, including a second car associated with a second player who is catching up from behind on the race track and attempting to overtake Player 1's car. The view recommendation engine evaluates the two camera views and identifies that AS1's active spectator camera view ASCV1.1 captures more interesting action in the game scene (i.e., the relative position of the second car to Player 1's car in the race) than the game scene captured by PCV1. As a result, the view recommendation engine may choose ASCV1.1 of AS1 instead of PCV1 to present to passive viewers.

[0051] In one embodiment, in addition to identifying specific camera views based on the context of the actions occurring within each camera view, the view recommendation engine may further filter camera views according to the passive viewer's profile. The passive viewer profile may specify that the passive viewer is interested in watching exciting action, selecting overhead views of game scenes, or following a specific player or active viewer, and the view recommendation engine identifies camera views according to the passive viewer's profile. In the above embodiment, if the passive viewer is interested in following player P1, the view recommendation engine may identify PCV1 instead of ASCV1.1 to present to the passive viewer. The passive viewer simply sits and enjoys the selected camera view identified according to their viewing preferences. As gameplay continues, the view recommendation engine dynamically switches between different player PCVs and different active viewer ASCVs, such switching correlates with the timing of different actions occurring in the video game.

[0052] Figures 3A and 3B show different perspectives of the gameplay data flow within the game cloud site 300 for identifying camera views to passive viewers. Camera views capturing different actions are streamed to passive viewers, and the streaming of each camera view correlates with the occurrence of different actions in the video game. The camera views constitute a portion of the recommendation stream provided to passive viewers.

[0053] Figure 3A shows a gameplay data flow used to generate various camera views in a video game in one embodiment, and Figure 3B shows a data flow for identifying a specific camera view from among the camera views generated in Figure 3A and providing it to a passive audience. The various camera views constitute parts of a recommendation stream provided to the passive audience.

[0054] Referring to Figure 3A, in one embodiment, multiple players access a game cloud site 300 via a network 200 such as the Internet and request to play a video game (e.g., Game 1). The request may include a game identifier for Game 1. The game may be a multiplayer game played by multiple players in the same or different locations. Requests from players are received and confirmed by the game cloud site 300, and after each player's confirmation is successful, one or more instances of the game are run on one or more game consoles 303 / game servers 302 in one or more data centers 301. Game inputs provided by multiple players during gameplay are processed by the game logic of Game 1 and used to update the game state of Game 1 and generate gameplay data 308. The game state of Game 1 resulting from the players' game inputs is synchronized across instances of the game running in one or more data centers 301. In some embodiments, each player playing the game may be part of a team competing against other teams within the game. Alternatively, each player may be playing independently of other players, encountering different players in different game scenes as the game progresses. In addition to the game inputs provided by the players, active spectators may provide their own input to select specific camera views they wish to view for each action occurring in Game 1. Active spectator input does not change the game state of Game 1. Active spectator input is processed by the game logic of Game 1, updating the gameplay data 308 of Game 1. The gameplay data 308 contains information used to generate different camera views for each action occurring in each game scene of Game 1. For example, the gameplay data 308 can be used to generate multiple PCV430a and multiple active spectator camera views (ASCV)430b. Each PCV430a is generated for a specific player and responds to the game input provided by that player. The identified PCV for each player is transferred to the player's client device and rendered. Based on the rendered PCV, each player may provide additional game input, which is processed by the game cloud site and generates additional gameplay data. This additional gameplay data is used by the view recommendation engine 400 to train the recommendation model. As more gameplay data is provided to the recommendation model, the model's output is fine-tuned. When gameplay data is requested from a passive viewer, the fine-tuning is used to identify a more appropriate camera view recommendation for that viewer. Similarly, based on the input provided by each active viewer, each ASCV430b is generated for that specific active viewer. The ASCV430b is returned to each active viewer during gameplay, allowing the active viewer to view the game scene in which the action is taking place according to their specific viewing preferences defined in their input.

[0055] Referring to Figure 3B, a passive viewer PS1 interested in watching gameplay of Game 1 accesses Game 1 running on the game cloud site 300 via the network 200. The game cloud site 300 identifies the passive viewer using the passive viewer's sign-on credentials and extracts the passive viewer PS1's profile 309. The passive viewer's profile 309 is provided as input to the view recommendation engine 400. The passive viewer's profile 309 contains the passive viewer's viewing preferences collected over a period of time. The view recommendation engine also receives various camera views generated from gameplay data 308 of game 1. The view recommendation engine 400 uses a machine learning algorithm 410 to generate a recommendation model, which is an artificial intelligence model, using information related to the camera views generated from the gameplay data as input. To generate different outputs that suit different viewing purposes specified by active viewers, the recommendation model is further trained using gameplay data collected from ongoing gameplay sessions. Outputs to be presented to passive viewers are identified from the recommendation model. Using the outputs from the recommendation model, a specific camera view is selected from among the camera views for each action that occurs in the video game. As each action occurs in the game scene of the video game, the camera view is selected based on the context of the action occurring in each camera view. The identified camera view may be a player camera view or an active viewer camera view.

[0056] When multiple camera views are identified based on the context of an action, the view recommendation engine 400 further filters the camera views using the viewing preferences defined in the passive audience PS1's profile. For example, two or more of the active audience camera views may capture a view of an action occurring in the video game. In this embodiment, the view recommendation engine 400 uses the passive audience PS1's viewing preferences to select a camera view from the active audience camera views to present to the passive audience PS1. Viewing preferences within a profile identify the passive viewer's viewing purpose and are identified based on the passive viewer's PS1 viewing history collected over time, which may be related to Game 1 and / or other game / interactive applications. A selected camera view that captures the view of the actions occurring in the video game's game scenes is streamed to the passive viewer's PS1. The various camera views provided to the passive viewer's PS1 during video game gameplay define the recommendation stream 430. The recommendation stream for the passive viewer's PS1 is specific to the passive viewer. Thus, different passive viewers may be provided with different camera views in their respective recommendation streams, and the camera views for different passive viewers follow the context of the actions occurring within the camera view and the respective passive viewer's viewing preferences.

[0057] The camera views identified in the recommendation stream 430 for the passive audience PS1 are then transferred to the passive audience's client device 100 by the view recommendation engine 400 and rendered. Thus, the view recommendation engine 400 can analyze the camera views of the game to identify specific camera views from among the camera views of the game scenes that are more meaningful to the passive audience, in order to enrich the passive audience's game viewing experience.

[0058] A request from a passive audience PS1 may be to watch live-streamed gameplay or pre-recorded gameplay of a game being played by multiple players. For brevity, various embodiments will be discussed in relation to live-streamed gameplay sessions of a game, but can be extended to pre-recorded gameplay sessions. Furthermore, various embodiments can be applied not only to MMO games but also to content generated in real-world environments, which can be live-streamed or pre-recorded and used later for streaming and live commentary in film production, etc.

[0059] Details of the camera views identified for the passive audience PS1 are forwarded to the Stream Recommendation Engine (or simply referred to as the "Recommendation Engine") 420. The Recommendation Engine 420 generates a Recommendation Stream 430 using the selected camera views for each action in each game scene of the game. The generated Recommendation Stream 430 is a collection of selected camera views for different actions in different game scenes of the game. Camera views identified in the recommendation stream 430 are transferred to the client device of the passive audience PS1 via the network 200 and rendered. The camera views transferred to the passive audience PS1 may be a combination of player camera views of one or more players and / or one or more active audience camera views, capturing different views of different actions. By providing selected camera views from among the camera views capturing the game scene in which the action is taking place, based on the context of the action and, if necessary, according to the viewing preferences of the passive audience PS1, the passive audience can become more engaged with the game, thereby increasing their interest in the game.

[0060] Figure 4 shows details of a view recommendation engine 400 used to process gameplay data 308 generated during gameplay in one embodiment. As previously mentioned, the game may be a multiplayer game played by multiple players. The game state of the game is updated and gameplay data 308 is generated using game inputs provided by multiple players of the game. Different game camera views are generated using the gameplay data 308, which form part of a recommendation stream 430 identified for passive spectators of the game. The gameplay data includes game input 308a from each player. The game input 308a from each player is used to influence the outcome of the game. In addition to the game input 308a provided by the players, one or more active spectators can provide input 308b during gameplay to influence the generation of an active spectator camera view (ASCV) 430b. In response to input 308b from the active audience, the game logic reruns the game using the gameplay data 308 available at the time of receiving input 308b from the active audience, generating an ASCV430b of the actions occurring in the game scene. A different PCV430a captured for each action occurring in the game scene captures a different player's perspective on the action, and the ASCV430b generated for each action captures a different view and aspect than that captured by the corresponding PCV430a. The view of each action in the game scene captured by ASCV430b along with the PCV430a contains sufficient data to construct a three-dimensional representation of the game scene associated with the action.

[0061] To identify camera views that form part of a recommendation stream 430 returned to the passive audience of Game 1, gameplay data 308 is processed to extract image data 311 and telemetry data 312. Image data 311 provides details of various image features contained in each game scene accessed by one or more players in the video game. Telemetry data 312 captures gameplay details, including game inputs provided by the player, characteristics of each activity such as what the player attempted, what the player achieved, what the player failed at, the player's player attributes, the effect of the player's game inputs on the game state, and game features of the game, such as type, gameplay level, complexity, etc. Image data 311 and telemetry data 312 are transferred to a feature processing engine 401 for further processing. Image data 311 may be stored in the image data store 311a, and telemetry data 312 may be stored in the telemetry data store 312a.

[0062] The feature processing engine 401 extracts image data 311 from the image data store 311a and processes the image data 311 to extract various features of the images captured in each camera view of the gameplay data. The image features of the game scene may include the features of each game object included in the game scene (e.g., length, width, height, size, weight, color, static or dynamic properties), such as the number and position of each game object, the number, position, and features of non-playing characters, and the number, position, and features of the player's adversaries or allies. Furthermore, the image features may also include the capture angle, capture depth, and clarity of the captured image. The feature processing engine 401 also extracts telemetry data 312 from the telemetry data store 312a and processes the telemetry data 312 to extract gameplay features and attributes of players who provided game inputs that affected the gameplay. The gameplay details contained in the telemetry data 312 can be used to identify the complexity of the various game inputs provided by the player, and the results of those game inputs in the game scenes captured in each camera view.

[0063] Image features, game features, and player attributes extracted from gameplay data 308 during gameplay are provided as input to a machine learning algorithm 410. The machine learning algorithm includes classifiers 410a defined for different features and attributes extracted from the game's gameplay data. A recommendation model 410b is generated using classifiers 410a. The recommendation model 410b is trained using game inputs received from the player during gameplay of the video game, and additional game inputs collected over time. In one embodiment, game inputs are collected from multiple gameplay sessions. Training of the recommendation model is performed to fine-tune the attributes and features included in various nodes of the recommendation model and to strengthen the relationships defined by edges between pairs of consecutive nodes of the recommendation model. The recommendation model defines various outputs, each related to a player's specific game behavior and / or the active audience's viewing preferences. The output from recommendation model 410b is selected based on the context of the action occurring in each of the various camera views that capture each action. In response to a viewing request from a passive audience, the selected output identifies the camera view to offer to the passive audience. In addition to the use of the action's context to select a specific camera view from among the camera views, the passive audience's viewing preferences may also be used to identify an output from the recommendation model. The passive audience's viewing preferences can be obtained from the passive audience's profile 309, which may include the passive audience's preferred game types, action types, view types, etc. This information can be obtained from the passive audience's browsing history collected over a period of time. Camera views that satisfy the passive audience's viewing preferences are identified and returned to the passive audience.

[0064] In some embodiments, multiple camera views capturing the view of an action may be identified according to the context of the action and the passive viewer's viewing preferences. In such embodiments, the passive viewer's viewing preferences are weighted, and the camera views are further filtered based on these weighted preferences to identify a specific camera view that best matches the passive viewer's weighted preferences. A specific camera view is recommended to the passive viewer. Depending on the actions occurring in the game scene, in some cases, camera views are recommended to the passive viewer according to their viewing preferences, thereby enabling the passive viewer to obtain an optimal viewing experience. The output from the recommendation model 410b is provided to the recommendation engine 420, which identifies the selected camera view from among the camera views for each action. The selected camera view for an action occurring in the game scene is provided for the duration of the action, and when the action is completed, the camera view is dynamically switched to another camera view corresponding to the action occurring during dynamic switching. Camera views of different actions streamed to passive viewers define a recommendation stream 430 of gameplay content. The recommendation stream 430 includes one or more PCV430a and one or more ASCV430b. The sequence of camera views in the recommendation stream corresponds to the sequence of various actions occurring in the gameplay of the video game. In one embodiment, the recommendation stream 430 generated by the recommendation engine 420 may be unique to each passive viewer, as it may include one or more camera views according to each passive viewer's viewing preferences. Each passive viewer's recommendation stream 430 is returned to and rendered on the viewer's respective client device 100 upon request from the passive viewer.

[0065] Figure 5 shows various modules of a view recommendation engine 400 used to process gameplay data and generate a recommendation stream for each passive audience member who has requested gameplay data, according to one embodiment. The view recommendation engine 400 includes multiple module / process engines that process gameplay data and identify various features and attributes contained in the gameplay data. Modules included in the view recommendation engine 400 may include a telemetry feature extraction engine 402, an image feature extraction engine 403, a player attribute extraction engine 402a, an audience attribute extraction engine 402b, and a gameplay data extraction engine 402c. In addition to the aforementioned modules / engines, the view recommendation engine 400 may also include, to name a few, a feature processing engine 401, a machine learning algorithm 410 including a classifier (i.e., classification engine) 410a and a recommendation model 410b, and a recommendation engine 420.

[0066] During gameplay, game input generated on each player's client device is forwarded to one or more game servers 302 running instances of the game for further processing. The game logic uses the game input to update the game state and generate gameplay data 308. Using the gameplay data 308, multiple camera views are generated, which form part of a recommendation stream 430 returned to passive viewers. The multiple camera views capture different perspectives of actions occurring in different game scenes of the game. The camera views generated from the gameplay data 308 include player camera views (PCVs) that capture each player's perspective on each action occurring in each game scene of the game. In addition to the PCVs, multiple active viewer camera views (ASCVs) are generated using input provided by active viewers. Input from the active audience specifies the depth, direction, clarity, angle, and content that the active audience wants to capture for each action. The game logic uses the active audience input to rerun the video game and generate an active audience camera view (ASCV) for each action. PCV430a and ASCV430b are forwarded along with gameplay data 308 to the view recommendation engine 400 running on game server 302. The game server 302 running the view recommendation engine 400 may be the same game server 302 running instances of the game accessed by one or more players and / or audiences, or it may be another game server 302 that is communicatively connected to one or more game servers 302 running the game. A player can participate in gameplay by accessing a single instance of the game running on game server 302. Alternatively, a player may access different instances of the game running on different game servers 302. In such cases, the game state of the game is synchronized across all running instances of the game, so the current game state of the game can be presented to the player.

[0067] The communication connection allows the view recommendation engine 400 to receive gameplay data 308 as a camera stream along with the generated game camera view. The gameplay data 308 incorporates telemetry data 312, and the game camera view incorporates image data 311 of the gameplay of multiple players. The telemetry data 312 incorporates the complexity of the actions performed in the game based on game input from the players.

[0068] The telemetry feature extraction engine 402 is configured to process the telemetry data 312 contained in the game camera view. The telemetry data 312 includes player attributes defined based on each player's game input and gameplay skills, and game features obtained from the execution of the game. The telemetry feature extraction engine 402 includes a player attribute extraction engine 402a, an audience attribute extraction engine 402b, and a gameplay data extraction engine 402c. The player attribute extraction engine 402a is configured to extract player-related data from the telemetry data 312. Some of the player-related data that can be extracted includes data related to player ratings, player level, game progress, gameplay strategies adapted by each player, entertainment value related to gameplay and players, player action sequences, and unique actions performed by players. Each player's player rating data can be updated from rating inputs provided by other players, active audiences, content providers, etc. Player-related data is used to identify player attributes (e.g., popularity, skill level / proficiency level) of each player that contribute to the game's appeal. Player-related data may be stored in the user data store 305 and used to update player profiles included in user accounts 304.

[0069] The gameplay data extraction engine 402c is configured to extract data related to game features such as game level data, data related to player-performed activities including attempted activities, completed activities, and the number of attempts for each activity, as well as each player's game win data and game state data. Activities performed by the player's game input can be associated with actions performed in the game's game scenes. Game-related features are used to identify the game state, saved data about each player including game customizations performed by each player, and so on. Game-related features can be stored in the gameplay data store 307.

[0070] The audience attribute extraction engine 402b is configured to extract active audience-related data from the telemetry data 312. Some of the active audience-related data that can be extracted includes the number of active audiences following the game's gameplay, the number of active audiences following each player, active audience profiles including audience ratings, and inputs provided by active audiences. Inputs provided by active audiences may include camera view-related inputs, such as specific angles, depths, and directions for capturing game actions occurring in game scenes of the game. In addition to the aforementioned inputs, active audiences may also provide other game-related inputs, such as comments, ratings, chats, emails, and blogs related to the gameplay of one or more players. Game-related inputs can be used to identify the type of comment provided, the quality of the comment, and the audience's knowledge of the game. Active audience-related data can be updated in audience profiles and can also be used to update game features and / or player attributes (e.g., popularity, skill level, etc.) of each player who provides game input to the game. Active audience-related data may be stored in the user data store 305 separately from or together with player attributes.

[0071] The image feature extraction engine 403 is similarly configured to extract various image features from the image data 311. Using the image features extracted from the image data 311, PCV430a can be distinguished from ASCV430b within the gameplay view. The extracted image features may include data related to the capture angle, capture depth, capture direction, clarity of the image captured in each capture view, content captured in the image, the game object targeted by the player's game input, and the position and characteristics of the game object targeted by the game input. Data related to the image may be stored in the image data store 311a.

[0072] The player attributes extracted by the player attribute extraction engine 402a, the active audience attributes extracted by the audience attribute extraction engine 402b, the game features extracted by the gameplay data extraction engine 402c, and the image features extracted by the image feature extraction engine 403 are provided as input to the feature processing engine 401 for further processing. The feature processing engine 401 identifies the complexity of the gameplay and correlates the features and attributes identified in various camera views generated from the gameplay data to specific actions that occur in each game scene of the game. The processed features, attributes, actions, game scenes, and other game-related data are provided from the feature processing engine 401 as input to the machine learning algorithm 410. The machine learning algorithm 410 includes a classifier 410a defined using one or more image features, game features, player attributes, and / or active audience attributes. The recommendation model 410b is generated using the information contained in the classifier 410a. The recommendation model 410b is an AI model built and trained using various features and attributes extracted from the gameplay data 308. The recommendation model 410b includes multiple nodes and edges. Various nodes are added to the recommendation model 410b using the feature and attribute information defined by the various classifiers, and the interrelationships between the information contained in each node are defined using the edges between any pair of consecutive nodes. When additional game data is generated from the game's gameplay, the AI ​​model is trained with the additional game data. The interrelationships become stronger by fine-tuning the features and attributes contained in each node and the corresponding edges that define the interrelationships between each node using the additional game data. The machine learning algorithm 410 identifies various camera views to be presented to passive viewers interested in watching gameplay. The identified camera views may include one or more player camera views or active viewer camera views, and the camera view selected for passive viewers is based on the context of the actions occurring in the relevant player camera view and active viewer camera view.

[0073] The machine learning algorithm 410 also receives profiles of passive viewers 309 who are interested in watching gameplay of a game. The profiles of passive viewers 309 identify the passive viewers' viewing preferences derived from their viewing history. These viewing preferences define the passive viewers' viewing objectives. In some embodiments, the machine learning algorithm 410 may also use the passive viewers' viewing preferences to identify the output of an AI model that best suits the passive viewers' viewing objectives. The output from the AI ​​model is used to identify camera views based on the context of the actions captured in each camera view, as well as to identify various camera views that suit the viewer's preferences. The recommendation model may identify different outputs for different passive viewers, each output identifying a set of camera views of the game scene that captures the view of the actions that occur in the game. The set of camera views is further filtered to identify specific camera views that match the preferences of a particular passive viewer. The output from the AI ​​model is provided as input to the recommendation engine 420. The recommendation engine 420 uses the output from the AI ​​model to extract data related to the selected camera views from among the camera views, processes the data to generate frames of game scene content, and streams the game scene frames over the network 200 to client devices 100 associated with each passive viewer for rendering. The camera views provided to passive viewers form part of a recommendation stream that is streamed to passive viewers during gameplay, providing views of actions that align with the passive viewers' viewing preferences.

[0074] Figure 6 shows various modules within the feature processing engine used to process various gameplay features and player attributes extracted by each extraction engine in one embodiment. In one embodiment, the feature processing engine 401 includes, for example, a telemetry feature processing engine 404, an image data analyzer 405, and a score generation engine 406. The telemetry feature processing engine 404 processes telemetry data 312 extracted from gameplay data 308 provided by the game server 302. The telemetry feature processing engine 404 may include multiple submodules that process different telemetry data 312. For example, the player attribute processing engine 404a within the telemetry feature processing engine 404 may be used to process data related to player attributes extracted by the player attribute extraction engine 402a and stored in the user data store 305. The player attributes in the user data store 305 are updated when additional player attributes are detected or when changes to existing player attributes of a player are detected. Player attributes can be used to measure not only the game customization defined by the player, but also the player's play style within the game, the player's popularity, and the player's gameplay value (e.g., entertainment value, strategic value, player's skill level, etc.). Similarly, the audience attribute processing engine 404b within the telemetry feature processing engine 404 can be used to process audience attributes of active audiences extracted by the audience attribute extraction engine 402b and stored in the user data store 305. Active audiences are those who actively provide input during gameplay to specify the camera views they want to view for different actions. Audience attributes can be used to define or update additional player attributes and / or additional game features of the game.

[0075] The gameplay data processing engine 404c within the telemetry feature processing engine 404 may be used to process game features extracted by the gameplay data extraction engine 402c. Using the game features, the game state, each player's saved game data, game popularity from players and / or active audiences, and game complexity may be identified.

[0076] The image data analyzer 405 is used to analyze the image data 311 extracted by the image feature extraction engine 403 and stored in the image data store 311a to identify various features of the game scene. Features may relate to the content of the game scene, the actions that occur in the game scene, the activity(s) performed by different players in the game scene, and / or the quality / complexity of the game scene. The image data analyzer 405 may include multiple submodules that evaluate the various features contained in the image data 311. The evaluation of image data includes assigning an associated score to each identified image feature and using the given scores when defining a classifier 410a using the machine learning algorithm 410. Some of the submodules of the image data analyzer 405 include a content evaluation engine 405a, a review evaluation engine 405b, and an activity evaluation engine 405c for evaluating features of the image data 311. The content evaluation engine 405a is configured to evaluate image data and identify content captured within each game scene. Content may include game objects, non-player characters, scenery, challenges, adversaries, allies, etc., which provide the overall appearance of the game scene. Image data related to features may include feature identifiers, specific characteristics such as height, weight, length, width, depth, size, appearance, location, and physical features (e.g., dynamic or immobile, direction of movement, trajectory, speed, etc.). The image data analyzer's review evaluation engine 405b is configured to evaluate the popularity of a game scene based on factors such as the number of times a registered player has visited the game scene during gameplay, and the number and type of comments received from active viewers regarding the features identified in the game scene. The image data analyzer's activity evaluation engine 405c is configured to evaluate the various actions that occur in each game scene, and the activities performed by the player via game input in each game scene. The evaluation also identifies the frequency of each activity that occurs in each game scene, the type and amount of game input provided to perform each activity, the type of action, the duration of the action, etc. The review evaluation engine 405b and the activity evaluation engine 405c evaluate image data derived from game input from the player (e.g., activities, actions, etc.) and input from the active audience (e.g., comments, chats, reviews, type and frequency of active audience interaction, etc.). Input from the active audience can be used to update the corresponding player profile and / or the game features of the corresponding game scene. The activity evaluation engine 405c is configured to identify and evaluate both player-related activities and active audience-related activities.

[0077] Information from the telemetry feature processing engine 404 and the image data analyzer 405 is provided as input to the score generation engine 406. The score generation engine 406 is configured to generate relevant scores for different camera views based on the identification and evaluation of various features and attributes. For example, a content-based score 406a may be calculated based on the evaluation of the content captured in each camera view of the game scene. Similarly, an activity / action-based score 406b may be calculated for each camera view based on the activities performed by the player and the actions that occur in response to the activities in each game scene captured in each camera view. A player-based score 406c may be calculated for each camera view based on player attributes identified from the activities performed in the game scene captured by that camera view. A spectator-based score 406d may be calculated for each different camera view based on the input provided by the active spectator for each camera view of the game scene. The scores from the score generation engine 406 are associated with each camera view and are used while generating and refining the recommendation model (i.e., AI model) 410b. The scores for the various camera views of different game scenes are continuously updated based on game input received from the player and input from the active spectator.

[0078] The input provided by the active viewer specifies the game viewing preferences for each action that occurs in the game. Game viewing preferences for a game scene may be for capturing actions occurring within the game scene, and may specify the capture angle, depth, and direction of the actions. These viewing preferences may be specific to each action and may be based on the type of action occurring in the game scene. Game viewing preferences define the viewer's purpose and are used to generate specific active viewer camera views of the game.

[0079] Recommendation model 410b is refined using player game inputs and active viewer inputs collected regarding the game. This refinement helps to strengthen the interrelationships between various features and attributes identified from gameplay data. Recommendation model 410b identifies different outputs for different viewing purposes. When a passive viewer selects a game to watch, recommendation model 410b identifies outputs that satisfy the passive viewer's viewing purpose. Viewing purposes are derived based on the context of actions occurring in the game's game scenes, and in some cases, may also be based on the passive viewer's profile. The audience profile identifies the types of actions, the amount of detail in those actions, and the type of camera view (overhead view, side view, etc.) that passive viewers prefer when watching a game. Details of passive viewers' viewing preferences are collected from their viewing history, which may be related to the game application and / or any other interactive application. Using the output from recommendation model 410b, a camera view to present to the passive viewer is identified for each action occurring in the game scene. For example, one or more camera views to present to the passive viewer may be identified based on the actions occurring in the game scene. When multiple camera views are identified, the camera view of an action can be identified using the passive viewer's viewing preferences. The camera view identified for a passive viewer may be the same as or different from the PCV. For example, a passive viewer profile may indicate that the passive viewer typically prefers to view an overhead camera view of an action, and the passive viewer of an action may be presented with a preferred overhead camera view of the action. In another example, a passive viewer may want to be presented with different camera views for different types of action, such as an overhead view for a racing game, a side view for a basketball game, and an opposing view for a tennis game. In one embodiment, an opposing view may be defined as a view that captures an action (e.g., a served or returned ball) from the viewpoint of an opposing player, such as an opponent in a tennis game, while the PCV of the same action may capture a view of the ball moving towards the opposing player from the player's viewpoint.

[0080] In one embodiment, the active audience camera view generated from the active audience's input may be a direction-based camera view or an action-based camera view. The score generation engine 406 calculates various feature / user-based scores according to the type of ASCV generated. The scores calculated for different camera views are provided to the machine learning algorithm 410 so that it can identify camera views suitable for presentation to the passive audience, and the recommendation model 410b is further refined. During gameplay of the video game, different camera views are presented to the passive audience, each camera view corresponding to a change(s) of action occurring in the game scene and presented to the passive audience for the duration of the change. The camera views presented to the passive audience are dynamically switched to correspond to actions occurring in the game scene. The dynamic switching is performed without input from the passive audience and is designed to provide the passive audience with camera views that capture various actions occurring in the game, according to the actions occurring and according to the passive audience's profile. The camera views presented to the passive audience form part of the recommendation stream 430.

[0081] Even when the same action is occurring in a game scene, the recommendation algorithm may provide different camera views in each recommendation stream 430 to different passive viewers. The difference in camera views between one passive viewer and another may be due to different viewing preferences specified in their respective profiles. The recommendation model 410b is configured to identify a suitable output by considering the viewing preferences of different passive viewers. The output identified by the recommendation model 410b is forwarded to the recommendation engine 420. The recommendation engine 420 uses the information provided in the output to identify a specific camera view from among the camera views and returns it to the passive viewer. Because the camera view is aligned with the passive viewer's viewing preferences, the passive viewer receives a satisfying game viewing experience based on the camera view presented to them.

[0082] Figures 7-1, A-1 to C-1, show examples of switching camera views provided by the recommendation engine 420 to multiple passive viewers of a game, based on the output identified from the recommendation model. Figure 7-1, A-1, shows one example of a switching camera view provided to passive viewer 1. Switching camera views include combinations of player camera views (PCVs) and active viewer camera views (ASCVs). For example, a recommended switching camera view provided to passive viewer PS1 includes, firstly, the PCV of player with player identifier 35, the PCV of player with player identifier 731, followed by ASCV 16.1, the PCV of player 3, the PCV of player 42, ASCV 20.7, and the PCV of player 35. Various camera views are provided by dynamically switching existing camera views with new camera views in virtually real time during the rendering of the video game. Each camera view is rendered over a period of time before being switched, and this period correlates with the duration of the action occurring in each camera view. For example, PCV35 is rendered for 4.5 seconds, then dynamically switched and replaced with PCV731. PCV731 is rendered for 7.5 seconds of gameplay, from 4.5 seconds to 12 seconds, then switched and replaced with ASCV16.1. ASCV16.1 is rendered for 7 seconds, then switched and replaced with PCV3, and so on. In the example of the recommended switching camera view shown in A-1 of Figure 7-1, PCV35 is shown twice, once at the beginning of the sample and once at the end of the sample. This may indicate that the player 35's camera view captures at different points in time the view of image features and / or action in progress that the passive spectator PS1 wants to view, and which in some cases may suit the passive spectator PS1's preferences. The ASCV presented to the passive audience identifies the player's game scenes that the active audience is interested in and that are generated based on the active audience's input. For example, ASCV 16.1 corresponds to an active audience camera view 1 that captures the game actions of a game scene associated with player 16. Similarly, ASCV 20.7 corresponds to an active audience camera view 7 that captures the game actions of a game scene associated with player 20. Figure 7-1, A-2 shows the stream obtained from the camera views provided to the passive audience PS1 while the passive audience PS1 is watching the gameplay of a video game.

[0083] Figure 7-1, B-1 shows another example of a sample of switching camera views provided to a second passive audience PS2 in one embodiment. The recommended stream includes a collection of ASCV3.5, 73.1, and 16.1 at the beginning, followed by the PCV of player 3, the PCV of player 42, ASCV20.7, the PCV of player 3, and ASCV3.5. As shown in Figure 7-1, A-1, a specific camera view is repeated from among the camera views. The switching camera views generated for each passive audience can not only capture game scenes from different angles, but also capture the progression of actions occurring in the game scenes. Actions occurring in a game scene may or may not respond to game input from the player. Figure 7-1, B-1, shows one such example, where the first ASCV3.5 can capture the start of an action, and the last ASCV3.5 can capture the progression of the action in the game scene over time. In this embodiment, the first and last ASCV3.5 may capture game scenes of action from the same angle, while other ASCVs and PCVs may capture game scenes associated with action from different angles (including different heights, different depths, different modes, different directions, etc.). Figure 7B-2 shows a recommendation stream obtained from camera views provided to a passive audience PS2 while the passive audience PS2 is watching gameplay of a video game.

[0084] Figure 7-1, C-1 shows another embodiment of a switching camera view provided to a passive viewer PS3, which includes a PCV for player 35, followed by a series of ASCVs (ASCV35.7, ASCV6.1, ASCV318.3, ASCV42.8, ASCV20.7, ASCV35.9), and PCV35. In one embodiment, all camera views included in the switching camera view provided to the PS3 may capture the same game scene associated with one or more actions, and each camera view may present the action from a different viewpoint according to the viewer's viewing preferences. Figure 7-1, C-2 shows a recommendation stream obtained from the camera views provided to the passive viewer PS3 while the passive viewer PS3 is watching gameplay of a video game.

[0085] Figure 7-2, D-1 shows another embodiment of a switching camera view provided to a passive viewer PS4, which includes only PCVs of different players capturing views of one or more actions occurring in a game scene of the game. In this embodiment, it is shown that the PCV of player 35 is present at both the beginning and the end of the switching camera view. This may be to show the progression of actions occurring within the game scene. Figure 7-2, D-2 shows a recommendation stream obtained from the camera views provided to the passive viewer PS4 while the passive viewer PS4 is watching gameplay of a video game.

[0086] Figure 7-2, E-1 shows yet another embodiment of a switching camera view provided to a passive viewer PS5, which includes only ASCVs generated from active viewer input that capture one or more actions occurring in a game scene. Figure 7-2, E-2 shows a recommendation stream obtained from camera views provided to a passive viewer PS5 while the passive viewer PS5 is watching gameplay of a video game. The embodiments described above are provided as embodiments showing that camera views identified for a passive viewer based on the passive viewer's specified preferences may include any combination of camera views that capture different actions or different views of the same action.

[0087] The various embodiments described herein provide a method for improving the viewing experience of passive viewers by identifying and presenting a selected camera view from among the camera views of game scenes of gameplay generated from the gameplay of multiple players. The selected camera view is identified based on preferences specified by the passive viewer in their profile, and is not identified based on recommendations of the game's game logic that can provide camera views from the player's point of view, or based on the popularity of an action or camera view. Providing passive viewers with camera views relevant to them (i.e., according to their viewing preferences) in a game can lead to a more satisfying viewing experience for them and can lead to increased audience retention in the game. Increased audience retention can lead to increased revenue for game developers, game players, game sponsors, etc.

[0088] Figure 8 shows the operation flow of a method for improving the audience viewing experience in one embodiment. The method begins with running a video game on a game server, as shown in operation 810. The video game may be run in response to a gameplay request for the video game received from one or more players. Running the video game generates gameplay data from game inputs provided by the players during gameplay. Using the gameplay data, camera views are generated, which include a player camera view to be provided to each player and an active audience camera view to be provided to each active audience member who accesses the video game and watches the players' gameplay. The active audience camera view is generated based on active inputs received from the active audience, which may specify the active audience's viewing preferences. Active audience viewing preferences may include capture angle, capture depth, capture direction, capture clarity, etc. Therefore, the active spectator camera view generated for each active spectator is also referred to as the active spectator camera view. The game may be a multiplayer game played by multiple players who are in the same location or in different geolocations. Game inputs provided by each player are used to influence the game state of the game and generate gameplay data. Using the gameplay data, multiple camera views are generated that capture different aspects of the game scene. Using the various camera views generated for the game scene, a three-dimensional view of the game scene may be generated. Camera views generated from gameplay data contain frames of content that capture a view of one or more actions occurring in the game scene. Frames of content associated with a player's player camera view are live-streamed to the player.

[0089] As shown in operation 820, a request to view video game gameplay is received from a passive viewer. The request specifies the video game that the passive viewer is interested in viewing, but does not include any other inputs that affect the generation of camera views. As shown in operation 830, in response to the request from the passive viewer, a view recommendation engine running on the game server identifies player camera views and active viewer camera views that capture the view of the action occurring in the video game's game scene, and presents them to the passive viewer as requested. The player camera views and active viewer camera views related to the action are generated using gameplay data available at the time the action occurs. The view recommendation engine then dynamically switches between the player camera view and the active audience camera view based on the context of the actions occurring in the player camera view and the active audience camera view. This dynamic switching is performed automatically without requiring input from the passive audience. The camera view is provided to the passive audience depending on the type of action captured. In some embodiments, the camera view is also based on the passive audience's viewing preferences, which are collected from the viewing history included in the passive audience's profile.

[0090] Traditionally, game logic presented all camera views generated for each action in each game scene, and passive viewers had to examine all the camera views to identify the one that best captured the game scene according to their preferences. Alternatively, game logic could present a specific player's camera view by matching the player's profile with the passive viewer's profile. Such camera views are not optimal for passive viewers, as they may be interested in viewing different perspectives of the game scene where the action is taking place, rather than the camera views provided by the game logic. Passive viewers may not be interested in generating new camera views by providing input like active viewers, but they may be interested in viewing the camera view that best captures the action within the game scene.

[0091] Various embodiments of the disclosure discussed herein provide a passive audience with camera views of different actions occurring in the game, using camera views already generated for the player and active audience, eliminating the need for the passive audience to create their own camera views. The camera views provided to the passive audience are camera views that suit the passive audience's viewing preferences and enable the passive audience to have a satisfying game viewing experience by providing views of the actions within the game scene.

[0092] In some embodiments, passive viewers may be provided with a user interface for updating their viewing preferences in their profile. These preferences specify the passive viewer's viewing purpose. A view recommendation engine, in conjunction with game logic, selects camera views according to the passive viewer's viewing purpose. The user interface may include options that the passive viewer can select. The various embodiments discussed herein can be used for both live-streaming games and pre-recorded gameplay.

[0093] In addition to gameplay, various embodiments can be extended to capture real-world content and stream it to an audience. In one embodiment, real-world content may be captured by multiple users, who may be grouped into different teams. Each team of users may capture a real-world scene from a different angle and transfer the captured stream to an application cloud server to share with other users / audience members. The application cloud server, associated with a view recommendation engine, may process the captured content of the real-world scene to identify various camera views that capture the action within the real-world scene, select a camera view from among them based on the passive audience's preferences, and stream it to the passive audience. Similar to gameplay in a game, real-world scenes captured by multiple users may be live-streamed to passive viewers via application cloud servers, or recordings of real-world scenes may be stored on application cloud servers and used by passive viewers when they want to view pre-recorded videos of real-world scenes. In embodiments of capturing real-world content, active viewers may be viewers who provide inputs that affect the content being captured in the camera view. The input may take the form of feedback or instructions to the user capturing real-world content. An alternative embodiment may include receiving real-world content captured by multiple users, each capturing the content from a different angle / depth / direction, and using the input of an active audience to identify a selected camera view from among the camera views to present to a passive audience. The passive audience only provides a request to view the real-world content, and the view recommendation engine automatically identifies a specific camera view from among the camera views to provide to the passive audience. The passive audience can then sit back and relax and enjoy the optimal gameplay camera view selected for them.

[0094] Figure 9 shows an embodiment of an information service provider architecture. An information service provider (ISP) 902 delivers numerous information services to geographically distributed and connected users 900 via a network 950. An ISP can deliver only one type of service, such as stock price updates, or a variety of services, such as broadcast media, news, sports, and games. Furthermore, the services provided by each ISP are dynamic; that is, services can be added or removed at any time. Therefore, the ISP providing a particular type of service to a particular individual may change over time. For example, while a user is in their local area, they can receive services from a nearby ISP, and if the user moves to another city, they can receive services from a different ISP. The local ISP transfers the necessary information and data to the new ISP, so that user information "follows" the user to the new city, and data becomes closer to the user and easier to access. In another embodiment, a master-server relationship may be established between a master ISP that manages user information and a server ISP that interfaces directly with the user under the control of the master ISP. In another embodiment, as the client moves around the world, data is transferred from one ISP to another, so that the ISP that is better located to serve the user becomes the ISP that delivers these services.

[0095] ISP902 includes Application Service Providers (ASPs)906, which provide computer-based services to customers over a network (including, but not limited to, any wired or wireless network, LAN, WAN, WiFi, broadband, cable, fiber optic, satellite, cellular (e.g., 4G, 5G), the Internet, etc.). Software provided using the ASP model is sometimes called on-demand software or software as a service (SaaS). Standard protocols such as HTTP are used for simple forms that provide access to specific application programs (such as customer relationship management). The application software resides on the vendor's system and is accessed by users via a web browser using HTML, via vendor-provided dedicated client software, or via other remote interfaces such as thin clients.

[0096] Services delivered across wide geographical areas often utilize cloud computing. Cloud computing is a computing style in which dynamically scalable, often virtualized, resources are delivered as a service over the internet. Users do not need to be experts in the technical infrastructure of the "cloud" that supports them. Cloud computing can be categorized into different services such as Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). Cloud computing services often provide common business applications online, accessed through a web browser, but the software and data are stored on servers. The term "cloud" is used as a metaphor for the internet (using, for example, servers, storage, and logic), based on how the internet is depicted in computer network diagrams, and is an abstract concept of hiding complex infrastructure.

[0097] Furthermore, ISP902 includes a game processing server (GPS)908 used by game clients to play single-player and multiplayer video games. Most video games played over the internet operate via a connection to a game server. Typically, games use a dedicated server application that collects data from players and distributes the collected data to other players. This is more efficient and effective than a peer-to-peer configuration, but requires a separate server to host the server application. In another embodiment, the GPS establishes communication between players and their respective gameplay devices, exchanging information without relying on a centralized GPS.

[0098] A dedicated GPS is a server that operates independently of the client. Such servers typically run on dedicated hardware located in a data center, providing greater bandwidth and dedicated processing power. Dedicated servers are the preferred method for hosting game servers for most PC-based multiplayer games. Large-scale multiplayer online games run on dedicated servers typically hosted by the software company that owns the game title, allowing the dedicated server to control and update the content.

[0099] The broadcast processing server (BPS) 910 distributes audio or video signals to viewers. Broadcasting to a very narrow audience is sometimes called narrowcasting. The final stage of broadcast distribution is the method of transmitting the signal to listeners or viewers, which can be transmitted terrestrially to antennas and receivers, as in the case of radio or television stations, or through cable television or cable radio (or "wireless cable") via a station, or directly from the network. The Internet can also provide radio or television to recipients, particularly using multicast, which allows for the sharing of signals and bandwidth. Historically, broadcasts have been defined by geographical area, such as national or regional broadcasts. However, with the proliferation of high-speed internet, content can reach almost every country in the world, so broadcasts are no longer defined by region.

[0100] A Storage Service Provider (SSP) 912 provides computer storage space and related management services. SSPs also offer regular backups and archiving. By providing storage as a service, users can order more storage as needed. Another significant advantage is that SSPs include backup services, ensuring that users will not lose all their data even if their computer's hard drive fails. Furthermore, multiple SSPs can hold full or partial copies of user data, allowing users to access their data efficiently regardless of their location or the device used for data access. For example, a user can access personal files on their home computer as well as their mobile phone while on the go.

[0101] A communications provider 914 provides connectivity to users. One type of communications provider is an Internet Service Provider (ISP), which provides access to the Internet. ISPs connect customers using data transmission technologies suitable for delivering Internet Protocol datagrams, such as dial-up, DSL, cable modem, fiber, wireless, or dedicated high-speed interconnects. Communications providers may also provide messaging services such as email, instant messaging, and SMS text. Another type of communications provider is a Network Service Provider (NSP), which sells bandwidth or network access by providing direct backbone access to the Internet. Network Service Providers may consist of telecommunications companies, data carriers, wireless communication providers, Internet Service Providers, and cable television operators providing high-speed internet access.

[0102] Data exchange 904 interconnects several modules within ISP 902 and connects these modules to users 900 via network 950. Data exchange 904 may cover a small area where all modules of ISP 902 are in close proximity, or it may cover a large geographical area when different modules are geographically dispersed. For example, data exchange 904 may include high-speed Gigabit Ethernet (or even faster) or intercontinental virtual area networks (VLANs) within a data center cabinet.

[0103] User 900 accesses the remote service via a client device 984 (i.e., client device 100 in Figure 1) which includes at least a CPU, memory, display, and I / O. The client device may be a PC, mobile phone, netbook, tablet, game system, PDA, etc. In one embodiment, ISP 902 recognizes the type of device used by the client and adjusts the communication method to be employed. In other cases, the client device accesses ISP 902 using a standard communication method such as HTML.

[0104] Figure 10 shows components of an exemplary game server device 302 that can be used to perform various embodiments of the present disclosure. For example, Figure 10 shows an exemplary server system having hardware components suitable for training an AI model capable of performing various functions related to a video game and / or gameplay of a video game, according to one embodiment of the present disclosure. A block diagram of the server system includes a server device 302, which may incorporate, or be itself, a personal computer, server computer, game console, mobile device, or other digital device, each suitable for practicing embodiments of the present invention. Alternatively, the functions of the server device 302 may be performed by a physical server, or a virtual machine, or a container server. The server device 302 includes a central processing unit (CPU) 1002 that runs software applications and optionally an operating system. The CPU 1002 may consist of one or more homogeneous or heterogeneous processing cores.

[0105] According to various embodiments, the CPU 1002 is one or more general-purpose microprocessors having one or more processing cores. Further embodiments can be implemented using one or more CPUs having a microprocessor architecture particularly suited to highly parallel and computationally intensive applications, such as media and interactive entertainment applications, or applications configured to perform deep learning, content classification, and user classification. For example, the CPU 1002 may be configured to include a machine learning algorithm 410 (also referred to herein as an AI engine or deep learning engine) configured to assist and / or perform learning operations with respect to providing various functions (e.g., prediction, suggestion) related to video games and / or gameplay of video games. The machine learning algorithm 410 may include a classifier 410a configured to build and / or train an AI model (i.e., a recommendation model) 410b using inputs and interactions provided during video game gameplay. The AI ​​model 410b is configured to provide suggestions for improving audience engagement metrics of the video game and / or group engagement metrics of the video game gameplay. Furthermore, the CPU 1002 includes an analyzer 1040 configured to analyze the inputs and interactions to generate and train the AI ​​model 410b and provide analysis results. The trained AI model 410b provides outputs in response to a specific set of player inputs and audience interactions, the outputs depending on predetermined functions of the trained AI model 410b. To satisfy defined engagement criteria for a video game, the trained AI model 410b can be used to identify optimal suggestions for the player and / or game logic that improve audience engagement metrics. The analyzer 1040 is configured to perform various functions related to video games and / or video game gameplay, including analyzing the outputs from the trained AI model 126b for a given input (e.g., controller input, game state data, success criteria), and to provide suggestions.

[0106] Memory 1004 stores applications and data used by the CPU 1002. Storage 1006 provides non-volatile storage and other computer-readable media for applications and data, and storage 1006 may include fixed disk drives, removable disk drives, flash memory devices, and CD-ROMs, DVD-ROMs, Blu-ray®, HD-DVDs, or other optical storage devices, as well as signal transmission and storage media. User input device 1008 transmits player input and audience interaction from one or more players and audience members to server device 302. Examples of user input devices 1008 may include keyboards, mice, joysticks, touchpads, touchscreens, still recorders / cameras or video recorders / cameras, game controllers, and / or microphones. The network interface 1014 enables the server device 302 to communicate with other computer systems via an electronic communication network, which may include wired or wireless communication over local area networks and wide area networks such as the Internet. The audio processor 1012 is adapted to generate analog or digital audio output from instructions and / or data provided by the CPU 1002, memory 1004, and / or storage 1006. The components of the server device 302, including the CPU 1002, memory 1004, data storage 1006, user input device 1008, network interface 1010, and audio processor 1012, are connected via one or more data buses 1022.

[0107] The graphics subsystem 1013 is further connected to the data bus 1022 and other components of the server device 301. The graphics subsystem 1013 includes a graphics processing unit (GPU) 1016 and graphics memory 1018. The graphics memory 1018 includes display memory (e.g., a frame buffer) used to store pixel data for each pixel of the output image. The graphics memory 1018 may be integrated into the same device as the GPU 1016, connected to the GPU 1016 as a separate device, and / or implemented within memory 1004. Pixel data may be provided directly from the CPU 1002 to the graphics memory 1018. Alternatively, the CPU 1002 provides the GPU 1016 with data and / or instructions defining a desired output image, and the GPU 1016 generates pixel data for one or more output images based on this. The data and / or instructions defining the desired output image may be stored in memory 1004 and / or graphics memory 1018. In one embodiment, the GPU 1016 includes a 3D rendering function that generates pixel data for an output image from instructions and data defining the geometric structure, lighting, shading, texture, motion, and / or camera parameters of the scene. The GPU 1016 may further include one or more programmable execution units capable of executing shader programs. In one embodiment, the GPU 1016 may be implemented within an AI engine to provide additional processing power, such as AI or deep learning capabilities.

[0108] The graphics subsystem 1013 periodically outputs pixel data of an image from the graphics memory 1018 and displays it on the display device 1010, or projects it using a projection system (not shown). The display device 1010 may be any device capable of displaying visual information in response to signals from the server device 301, including CRT, LCD, plasma, and OLED displays. The server device 301 may provide the display device 1010 with, for example, analog or digital signals.

[0109] Embodiments of the present disclosure can be implemented in a variety of computer system configurations, including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, miniature computers, and mainframe computers. The disclosure can also be implemented in a distributed computing environment in which tasks are performed by remote processing devices linked via a wired or wireless network.

[0110] In some embodiments, communication can be facilitated using wireless technology. Such technologies may include, for example, 5G wireless communication technology. 5G is the fifth generation of cellular network technology. A 5G network is a digital cellular network, in which the service area targeted by a provider is divided into small geographical areas called cells. Analog signals representing voice and images are digitized by the telephone, converted by an analog-to-digital converter, and transmitted as a stream of bits. All 5G wireless devices within a cell communicate by radio waves using local antenna arrays and low-power automatic transceivers (transmitters and receivers) within the cell, and this communication takes place via frequency channels allocated by the transceivers from a pool of frequencies reused by other cells. The local antennas are connected to the telephone network and the internet by high-bandwidth optical fiber or wireless backhaul connections. As with other cell networks, mobile devices moving from one cell to another are automatically transferred to the new cell. It should be understood that 5G networks are merely one type of communication network, and embodiments of this disclosure may utilize not only subsequent generations of wired or wireless technologies following 5G, but also previous generations of wireless or wired communications.

[0111] With the embodiments described above in mind, it should be understood that this disclosure may use a variety of computer operations involving data stored in a computer system. These operations require the physical manipulation of physical quantities. Any of the operations described herein that form part of this disclosure are useful machine operations. This disclosure also relates to devices or apparatus for performing these operations. The apparatus may be specifically built for a required purpose, or it may be a general-purpose computer selectively enabled or configured by a computer program stored in the computer. Specifically, a variety of general-purpose machines may be used by a computer program written in accordance with the teachings herein, or it may be more convenient to build an apparatus more specialized to perform the required operations.

[0112] This disclosure can also be embodied as computer-readable code on a computer-readable medium. Alternatively, the computer-readable code may be downloaded from a server using the aforementioned data exchange interconnect. The computer-readable medium is any data storage device capable of storing data that can then be read by a computer system. Examples of computer-readable mediums include hard drives, network-attached storage (NAS), read-only memory, random-access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. The computer-readable medium may include computer-readable tangible media distributed across network-connected computer systems so that the computer-readable code is stored and executed in a distributed manner.

[0113] While the method operations were described in a specific order, it should be understood that, as long as the overlay operations are processed in the desired manner, other maintenance operations may be performed between operations, or operations may be coordinated to occur at slightly different times, or operations may be distributed throughout the system to allow processing operations to occur at various processing-related intervals.

[0114] While the foregoing disclosure has been described in some detail for clarity, it will be apparent that certain changes and modifications can be made within the scope of the appended claims. Accordingly, this embodiment should be considered illustrative rather than restrictive, and this disclosure should not be limited to the details provided herein, but may be modified within the scope of the embodiments described and their equivalents.

[0115] It should be understood that the various embodiments defined herein may be combined or assembled into a particular embodiment using the various features disclosed herein. Therefore, the provided embodiments are only a few possible embodiments and are not limited to the various embodiments that can be defined by combining various elements. In some embodiments, some embodiments may include fewer elements without departing from the spirit of the disclosed embodiments or equivalent embodiments.

Claims

1. In response to a gameplay request received from a player, the game cloud server executes the video game, and the execution of the video game generates gameplay data from game inputs provided by the player, and the execution of the video game processes the gameplay data to generate a player camera view to be provided to the player during gameplay, and an active audience camera view to be provided to active audiences who access the video game and watch the player's gameplay, based on the active audience's input. The system receives a request from a passive audience to view the video game running on the game cloud server. In response to the viewing request, the presenter presents either the player camera view or the active audience camera view, the presenter includes dynamic switching between the player camera view and the active audience camera view of the gameplay provided to the passive audience, the dynamic switching being a method based on the context of the actions occurring in the player camera view and the active audience camera view. The dynamic switching is performed without input from the passive audience, and the operation of the method is performed by the game cloud server. The dynamic switching is based on the profile of the passive audience. The profile of the passive audience identifies the passive audience's viewing preferences, which are collected over time based on the content the passive audience has viewed. In the dynamic switching described above, the player camera view or the active spectator camera view in the gameplay is switched from a first player camera view or the first active spectator camera view to a second player camera view or the second active spectator camera view in the gameplay that is different from the first player camera view or the first active spectator camera view. method.

2. The method according to claim 1, wherein the player camera view and the active spectator camera view are dynamically updated during gameplay to capture a view of the actions occurring in the video game.

3. The method according to claim 1, wherein the dynamic switching correlates with the time at which a change is detected in the action occurring in the video game.

4. The method according to claim 1, wherein the dynamic switching is performed when no change is detected in the action within the video game for a predetermined period of time, and the dynamic switching is correlated with the expiration of the predetermined period of time.

5. The method according to claim 1, wherein the dynamic switching to the player camera view or the active spectator camera view is performed in real time.

6. The method according to claim 1, wherein the viewing preferences of the passive audience are dynamically adjusted in the profile based on preference input received from the passive audience.

7. The method according to claim 1, wherein the actions occurring in the video game are based on the activities performed by the player, and the player camera view and the active spectator camera view capture the actions related to the activities.

8. As the dynamic switching continues during the gameplay of the video game, a recommendation stream is generated to be provided to the passive audience. The method according to claim 1, wherein the recommendation stream includes a combination of one or more player camera views and / or one or more active spectator camera views, each player camera view and each active spectator camera view being dynamically generated and including a separate view of the action occurring in the video game.

9. The method according to claim 1, wherein a plurality of actions occur during gameplay of the video game, each of the plurality of actions is associated with one or more player camera views and one or more active spectator camera views, the one or more player camera views and the one or more active spectator camera views associated with each action capture a separate view of the corresponding action occurring in the game scene, and the one or more player camera views and the one or more active spectator camera views generated with respect to the corresponding action include sufficient data to construct a three-dimensional representation of the game scene of the video game associated with the action.

10. In the processing of the aforementioned gameplay data, The method according to claim 1, wherein the video game is rerun to generate an active audience camera view that captures the actions occurring in the video game at a desired angle specified by the input of the active audience, the rerun is performed using the gameplay data available at the time the input from the active audience is received.

11. The player camera view captures a view of the actions occurring in the video game from the player's point of view. The method according to claim 1, wherein the active audience camera view captures a view of the action that is different from the view captured by the player camera view, and the active audience camera view is generated based on the context of the action and the input provided by the active audience.

12. The method according to claim 11, wherein the action that occurs in the video game occurs in response to game input provided by the player.

13. The method according to claim 1, wherein the dynamic switching includes presenting the player camera view selected for the passive audience or the active audience camera view for the duration of the action occurring in the video game.

14. The gameplay data is aggregated gameplay data that includes gameplay data generated from the gameplay of multiple players currently playing the video game, and using the aggregated gameplay data, player camera views of the multiple players and active spectator camera views of multiple active spectators are generated for each action that occurs in the video game. Each player camera view captures a view of the video game from the viewpoint of a specific player using the game input of that specific player among the plurality of players, and each active spectator camera view captures a view of the video game according to the view specifications provided by the input of a specific active spectator among the plurality of active spectators. The method according to claim 1, wherein the dynamic switching is performed to provide views of game scenes associated with different actions occurring in the video game, wherein the switching is performed to (a) only the player camera view, or (b) only the active spectator camera view, or (c) the player camera view and the active spectator camera view.

15. The method according to claim 13, wherein the dynamic switching is further based on the profile of the passive audience.

16. In the aforementioned dynamic switching, further, A recommendation model is generated using machine learning logic, and the recommendation model is generated and trained using gameplay data from the video game, and the gameplay data is generated in response to the player's game input and the active audience's input. The method according to claim 1, wherein an output correlated with the context of an action occurring in the video game is identified from the recommendation model, and the player camera view or the active spectator camera view for the dynamic switching is identified using the output.

17. The system detects viewing requests for video games running on a game cloud server, and these requests are received from passive viewers. Identify a plurality of camera views generated during gameplay of the video game, wherein the plurality of camera views include player camera views generated for a plurality of players and active audience camera views generated based on the input of an active audience for a plurality of active audiences following the gameplay of the video game, and each player camera view and each audience camera view captures a separate view of the game scene of the video game associated with each action that occurs in the video game. A method comprising, in response to the request, presenting to the passive audience a selected camera view from the plurality of camera views, either the player camera view or the active audience camera view, for each action, wherein the presentation includes dynamic switching of the selected camera view from the player camera view or the active audience camera view, The dynamic switching is performed without input from the passive audience and is based on the context of each action occurring in the respective player camera view and audience camera view associated with each action. The operation of the above method is performed by a server of the game cloud system. The dynamic switching is based on the profile of the passive audience. The profile of the passive audience identifies the passive audience's viewing preferences, which are collected over time based on the content the passive audience has viewed. In the dynamic switching described above, the player camera view or the active spectator camera view in the gameplay is switched from a first player camera view or the first active spectator camera view to a second player camera view or the second active spectator camera view in the gameplay that is different from the first player camera view or the first active spectator camera view. method.

18. The method according to claim 17, wherein the selected camera view is identified from among the player camera view or the active spectator camera view for the dynamic switching, correlates with the time at which a change is detected in each action occurring in the video game, and is presented over the duration of each action.

19. The method according to claim 17, wherein each action occurring in the video game occurs in response to game input provided by one or more of the plurality of players, the game input causes a change from a first game scene to a second game scene, and the plurality of camera views generated for each action capture a view of the second game scene.

20. The player camera view generated for one of the aforementioned players captures a view of the game scene associated with each action occurring in the video game from the player's point of view. The method according to claim 17, wherein an active spectator camera view generated for an active spectator captures a view of the game scene associated with each action, which differs from the view captured by the corresponding player camera view, the input from the active spectator identifies a desired view direction of the game scene for each action specified by the active spectator.

21. The active audience camera views for each of the multiple active audiences for each action are: The method according to claim 17, wherein the generated active audience camera view is generated by re-running the video game for each action using the gameplay data available at the time each active audience input is received, and the generated active audience camera view captures a view of the game scene associated with each action, which is different from the view captured in the corresponding player camera view.

22. The method according to claim 21, wherein the re-run and dynamic switching of the video game are continued during the gameplay of the video game or while the passive viewer selects to watch the video game.

23. The method according to claim 17, wherein each player camera view generated during gameplay and associated with each action is specific to one of the plurality of players, and each player camera view is generated to capture a view of a corresponding action performed in response to game input provided by the player.

24. As the dynamic switching continues during the gameplay of the video game, a recommendation stream is generated to be provided to the passive audience. The method according to claim 17, wherein the recommendation stream includes a combination of one or more player camera views and / or one or more active audience camera views.

25. The method according to claim 17, wherein the dynamic switching includes presenting the player camera view and the active spectator camera view associated with each action for the duration of the action.

Images