Server-Side Game Environment Recreation for Multi-Perspective Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video game streaming technologies lack the ability to effectively display and represent actions happening to other players in separate parts of a game or eSports competitions, limiting the commentary and analytics capabilities of streamers.
Innovation Solution
A system that receives user action data from clients, generates a recreated game environment at a server, and transmits a video stream from a selected viewing position, allowing for commentary and analytics similar to those performed by analyst teams in physical sports, including automated broadcast studio functionality and video queue management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If streamers show live footage of their own gameplay only, then the streaming setup remains simple, but the ability to display and represent actions happening to other players in separate parts of the game is lost
Solution Approach 1:
The system creates virtual copies of the game environment from the same game state data, allowing multiple simultaneous perspectives without requiring multiple physical game instances. The server generates synthetic video feeds from different camera angles by rendering the same game state from multiple virtual camera positions, enabling streamers to show other players' experiences while maintaining a single game instance.
Solution Approach 2:
A server acts as an intermediary between the game client and the streaming system. The client sends game state data to the server, which then processes this data to generate multiple perspective views. This intermediary architecture allows complex multi-perspective rendering without burdening the client hardware or requiring multiple game installations.
2Loss of information
If multiple game instances are run to show different players' perspectives, then all player actions can be displayed, but the resource consumption and system complexity increase significantly
Solution Approach 1:
Instead of running multiple physical game instances, the system creates virtual copies of the game environment from a single game state data source. The server receives game state information from one client and generates multiple rendered perspectives from this same data, eliminating the need for multiple game installations while maintaining complete coverage of all player actions.
Solution Approach 2:
A single game instance serves multiple functions by providing game state data that is reused to generate multiple different video perspectives. The same game state information is processed to create views from different players' perspectives, commentator views, and highlight reels, maximizing the utility of one game instance.
3Measurement precision
If streamers manually capture and edit gameplay footage, then commentary quality can be maintained, but real-time analytics and highlight reel generation become time-consuming
Solution Approach 1:
The system performs preliminary processing of game state data by continuously analyzing player actions, game events, and statistics in real-time. Highlight-worthy moments are identified and segmented automatically as they occur, and multiple perspective views are pre-rendered from the game state data, so that when streamers need footage, it is already prepared and organized for immediate use.
Solution Approach 2:
The system provides real-time feedback to streamers by analyzing game state data and automatically identifying highlight moments, player statistics, and noteworthy events. This feedback mechanism suggests relevant footage segments and analytics data to streamers, enabling them to maintain accurate commentary without manually reviewing all gameplay footage.
Data Source
AI summary
User action data characterizing action by a player in a game environment executing at a user client is received at a server. The game environment is created by the user client separate from the server. Data characterizing a selected viewing position is received. The selected viewing position is different than a player viewing position. The selected viewing position characterizes a viewing location within the game environment. A recreated game environment is generated from the user action data at the server. A video stream of the recreated game environment is generated. The video stream includes video from a perspective of the selected viewing position. The video stream is transmitted to a viewing client. Related apparatus, systems, articles, and techniques are also described.