Automatically generating enhanced activity and event summaries for game sessions

Abstract

The present disclosure relates to automatically generating enhanced activity and event summaries for game sessions. Game summaries are generated based on game data associated with a game play session using event logs of in-game events and corresponding game content. The event logs can include metadata indicating a time of an in-game event and an association between the in-game event and a game content item that captures the in-game event. Users can interact with the in-game events through a temporal context, making more informed choices and better understanding the game session. Using the event logs, the game summaries can provide features such as timelines that convey the relative timing of in-game events, lists of in-game events, maps of the game's virtual environment time annotated based on in-game events, game state information, and statistics and performance information. The game summaries can show trends over time and / or game sessions and convey information such as selected player groups of teams.

Description

BACKGROUND

[0001] Staying in touch with others through social media, texting, or email has become an integral part of the way society functions today. In some popular social media applications, a user is presented with an interface that displays posts, videos, and other (usually) curated information in the form of an activity feed. The activity feed can be used to assess what those within a user’s network are engaging in. Video games have become an increasingly popular source of social connection for people to interact with their friends or connect online with others who have similar gaming interests. Often, users maintain a list of friends to facilitate interaction with one another during game play sessions and to establish social connections with other users. Historically, however, video games have had a limited presence in activity feeds, often just as a way to secretly or openly encourage others to play games.

[0002] Some gaming systems provide a basic activity feed for games. Users can manually take screenshots or clips of game play sessions using a game operating system (e.g., a game operating system of a console) and share them with other users within their gaming social circle. These posts can come with a brief description of the game that was captured, but often do not provide much context for the viewer to understand what happened in the game play session. In some cases, achievements unlocked by a player during game play can trigger corresponding posts in the player’s social feed. However, utilizing these features requires game developers to specifically code for social feed interactions in the game, which not all developers can invest the time and resources to do. Moreover, these features are implemented using the application programming interface (API) of the gaming system, which limits the publishing functionality to those supported by the game engine and game platform. Furthermore, in traditional gaming platforms that include an activity feed, social posts about a game play session cannot be published once the game play session is complete. Additionally, determining whether a particular section of a video or video clip contains interesting content (e.g., more action) is important for deciding which videos might be worth including in the activity feed such that “interesting content” is automatically captured for review and sharing at a later time. SUMMARY

[0003] The present disclosure relates to automatically generating and updating enhanced summaries of activities and events of a game play session. In an embodiment, a game summary can be generated based on metadata extracted from game data of a game play session including in-game events. The present disclosure relates to systems and methods for generating game summaries using metadata that collect and communicate in-game events occurring within one or more game play sessions so that viewers can access associated screenshots, video clips, and / or other game content to better understand the one or more game play sessions.

[0004] Unlike conventional systems, a game summary can be generated using an event log of in-game events and corresponding screenshots, video clips, and / or other game content, each of which can be automatically generated based on analyzing video data, user input data, game data, data sent by the game using an application programming interface (API), and / or other data related to the game play session. The event log can include metadata indicating the time of the in-game event within the game play session, as well as the association between the in-game event and the game content item that captured the in-game event. In some embodiments, using the disclosed method, a user can interact with in-game events with a temporal context, thereby allowing for more informed choices and a better understanding of the game play session. In various embodiments, an enhanced game summary can be generated using the event log, which provides features such as a timeline conveying the relevant timing of in-game events, a list of in-game events (e.g., thumbnails), a map of the game virtual environment annotated over time based on in-game events, game state information, and statistical and performance information. One or more sections of the game summary can correspond to one or more game sessions and / or players, for example, displaying trends over time and / or game sessions, and conveying information about a selected group of players (e.g., a team). BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The following describes in detail the systems and methods of the present disclosure for generating automatic game summaries for game play sessions with reference to the accompanying drawings, wherein:

[0006] Figure 1 An example system diagram illustrating a game summary system according to some embodiments of the present disclosure is shown;

[0007] Figure 2 An example of a game summary of an in-game event is shown, including a map of one or more virtual environments in which the in-game event occurs, according to some embodiments of the present disclosure;

[0008] Figure 3 An example of a game summary of in-game events is shown, including a game content library capturing one or more in-game events, according to some embodiments of the present disclosure;

[0009] Figure 4 shows examples of additional information that may be included in a game summary of in-game events according to some embodiments of the present disclosure;

[0010] Figure 5 A flow chart showing a method for presenting a game summary including a timeline associated with events in the game is shown according to some embodiments of the present disclosure;

[0011] Figure 6 A flowchart illustrating a method for presenting a game summary including a list associated with in-game events according to some embodiments of the present disclosure is shown;

[0012] Figure 7 A flowchart illustrating a method for presenting a game summary including a map of a virtual environment associated with an in-game event according to some embodiments of the present disclosure is shown;

[0013] Figure 8 An example system diagram of a game streaming system according to some embodiments of the present disclosure is shown;

[0014] Figure 9 is a block diagram of an example computing environment suitable for implementing some embodiments of the present invention; and

[0015] Figure 10 is an example data center suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0016] The present disclosure relates to automatically generated game summaries for game play sessions. In embodiments, a game summary can be generated based on metadata extracted from game data of a game play session including in-game events. The present disclosure relates to systems and methods for generating game summaries using metadata that convey in-game events that occurred in one or more game play sessions so that viewers can access, view, and curate (e.g., for posting on a social network) associated screenshots, video clips, and / or other game content to better understand the one or more game play sessions.

[0017] A game summary can be generated using an event log of in-game events and corresponding screenshots, video clips, and / or other game content, each of which can be generated based on analyzing video data, user input data, game data, data sent by the game using an application programming interface (API), and / or other data related to the game session. The event log can include metadata indicating the time of the in-game event within the game play session, as well as the association between the in-game event and the game content item that captured the in-game event. In certain embodiments, using the disclosed method, a user can interact with in-game events with a temporal context, thereby allowing for more informed choices and a better understanding of the game play session. One or more parts of the game summary can be displayed in the user's activity feed and / or can be accessed from the activity feed, thereby providing a more robust view of player activity. In one or more embodiments, one or more parts of the game summary can correspond to one or more game sessions and / or players, such as displaying trends over time and / or game sessions, and conveying information about a selected set of players (e.g., a team).

[0018] In at least one embodiment, using the event log, the game summary can provide an interface that displays interface elements (e.g., including thumbnails) corresponding to or otherwise indicating events in the game. In certain embodiments, a timeline can also be displayed to convey the relevant timing of events in the game. In at least one embodiment, the interface elements can form a list of events in the game, and metadata can be used to display them in a time sequence (e.g., arranged in the order of occurrence). For example, each interface element can correspond to a thumbnail of a video clip or screenshot in which one or more in-game events occur, and metadata can be used (e.g., in an image carousel) to display them in chronological order. In at least one embodiment, a user can select an interface element, which can cause the relevant game content item capturing the event in the game to be loaded into the user interface.

[0019] In embodiments including a timeline, the timeline may also be updated to indicate in-game events based on selections. For example, the update may include visual emphasis of at least an icon or other indicator on the timeline representing an in-game event. Additionally or alternatively, the event log may organize in-game events into rounds or matches within a game play session. When a user selects an interface element, the timeline may be updated to correspond to a round including an in-game event (e.g., displaying the round in the timeline and / or matching the timeline to the in-game event for the round). Game state information displayed in the user interface may also be updated to correspond to the in-game event. For example, the event log may be used to update one or more player scores to reflect their status during the game associated with the in-game event.

[0020] In a further aspect, the user interface may include a map of the game virtual environment corresponding to the game play session. Based on the selection of the interface element, the map may be annotated with the location and time associated with the in-game event in the metadata. The location may correspond to the player and / or another in-game object at the time and / or during or after the in-game event occurs. In various embodiments, the annotation may include displaying and / or updating the path of the in-game object on the map, where the destination may be based on the location and time. For example, a path or other indicator of one or more locations of a player, enemy, non-player character (NPC), in-game event, item, and / or other element may be displayed on the map to reflect the game state before, during, and / or after the in-game event. In various embodiments, the elements may be displayed using corresponding symbols or icons, and different symbols or icons may be used for each element type.

[0021] In at least one embodiment, game data associated with a game play session may be analyzed to automatically detect in-game events, which may trigger the recording and / or saving of game play via screenshots, videos, and / or other game content items that capture game events and metadata (e.g., timestamps, in-game object locations, scores, player states, player statistics, game participants by username, event types, kill / death / assist ratios, etc.) The analysis may be performed during the game play session (e.g., by analyzing one or more aspects of the game play (e.g., video, user input, etc.), API messages from the game, and / or streams from viewers of a video streaming platform or application), and / or the analysis may be performed after the game play session (e.g., on video files and / or saved replays of the game play session or other game data).

[0022] In various embodiments, an algorithm that observes the screen for visual cues can be used to detect in-game events (e.g., using neural networks, computer vision, etc.). When these visual cues are detected, it can automatically trigger the recording and / or saving of game play. In at least one embodiment, it can additionally or alternatively trigger further analysis of the corresponding video data to extract one or more elements of metadata, or at least some metadata can be derived by detecting the corresponding visual cues. Metadata, screenshots, video clips, maps (e.g., extracted from and / or associated with the game), player icons in the game, etc. can be stored in the data entity for use in displaying a game summary.

[0023] refer to Figure 1 , Figure 1is an example system diagram of a game summary system ("GSS system") 100 according to some embodiments of the present disclosure. It should be understood that this and other arrangements described herein are set forth by way of example only. Other arrangements and elements (e.g., machines, interfaces, functions, commands, functional groupings, etc.) may be used in addition to or in place of the arrangements and elements shown, and some elements may be omitted entirely. In addition, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or in combination with other components, and in any suitable combination and location. The various functions described herein as being performed by the entities may be performed by hardware, firmware, and / or software. For example, the various functions may be performed by a processor executing instructions stored in a memory.

[0024] GSS system 100 may include, among other things, client devices 104(A), 104(B), and 104(C) (collectively referred to herein as "client devices 104"), game summary server 116, and / or game server 126. Figure 1 104(A), 104(B), and 104(C) are shown, but this is not intended to be limiting. In any example, there may be any number of client devices 104. GSS system 100 (and its components and / or features) may be implemented using one or more computing devices.

[0025] Components of GSS system 100 may communicate via networks 102. Networks may include wide area networks (WANs) (e.g., the Internet, the public switched telephone network (PSTN), etc.), local area networks (LANs) (e.g., Wi-Fi, ZigBee, Z-Wave, Bluetooth, Bluetooth Low Energy (BLE), Ethernet, etc.), low power wide area networks (LPWANs) (e.g., LoRaWAN, Sigfox, etc.), global navigation satellite system (GNSS) networks (e.g., the Global Positioning System (GPS)), and / or other network types. In any example, each component of GSS system 100 may communicate with one or more other components via one or more networks 102.

[0026] Client devices 104 may include smartphones, laptops, tablets, desktop computers, wearable devices, game consoles, virtual reality systems (e.g., headsets, computers, game consoles, one or more remote controls, one or more controllers, and / or other components), streaming devices, smart home devices that may include intelligent personal assistants, and / or other types of devices capable of supporting game play.

[0027] The client devices 104 can include one or more applications 106, a display 108, a communication interface 110, one or more input devices 112, a graphical interface manager 130, a game data capturer 138, and an interest determiner 140. The one or more game summary servers 116 can include a graphical interface manager 130, a communication interface 122, and one or more data stores 124. The one or more game summary servers 116 can include one or more portions of the graphical interface manager 130, the game data capturer 138, and / or the interest determiner 140 in addition to, or instead of, those components included in the one or more client devices 104. The one or more game servers 126 can include a game engine 128, a communication interface 132, and a data store 134.

[0028] Although Figure 1 Only some components and / or features of the client devices 104, the game summary servers 116, and the game servers 126 are illustrated in the figures, this is not intended to be limiting of the present disclosure. For example, the client devices 104, the game summary servers 116, and the game servers 126 can include additional or alternative components. Moreover, the configuration of the components is provided by way of example only, but can be highly flexible depending on the implementation of the GSS system 100. For example, one or more of the game servers 126 can be implemented as one or more game summary servers 116 and / or include at least some of the functionality thereof, as described herein. Examples include one or more portions of the graphical interface manager 130, the game data capturer 138, or the interest determiner 140. Similarly, at least some of the functionality of the game summary servers 116 can be included in the applications 106, can not be included in the applications 106, or can be executed at least in part by the client devices 104 or not by the client devices 104. Figure 1 The examples are indicated by the graphical interface manager 130, the game data capturer 138, and the interest determiner 140 in the game summary servers 116 and the client devices 104 in the figures.

[0029] As an overview, the applications 106 may include any of a variety of potential types of software capable of presenting one or more game summaries (e.g., game summaries 136) on the display 108 of the client device 104. In at least one embodiment, the applications 106 (and / or different applications on one or more client devices 104) may include a gaming application that facilitates playing games (e.g., cloud and / or local games) on the client device 104 via one or more input devices 112. The communication interface 110 may include one or more components and features for communicating across one or more networks (e.g., one or more networks 102), such as receiving and / or sending data corresponding to one or more game summaries (e.g., metadata, video clips, screenshots, etc.), user input to the input devices 112, streaming media content, etc.

[0030] The input device 112 may include any type of device capable of providing user input to the game. The input device may include a keyboard, a mouse, a microphone, a touch screen display, a controller, a remote control, a headset (e.g., a sensor for a virtual reality headset), and / or other types of input devices.

[0031] The communication interface 110 may include one or more components and features for communicating across one or more networks (e.g., one or more networks 102). The communication interface 110 may be configured to communicate via any number of networks 102 described herein. For example, to Figure 1 In order to communicate with the GSS system 100 of the present invention, the client device 104 can use an Ethernet or Wi-Fi connection through a router to access the Internet to communicate with one or more game summary servers 116, one or more game servers 126 and / or other client devices 104.

[0032] The graphical interface manager 130 may include any of a variety of computer-readable media. Computer-readable media may be any available media that can be accessed by the client device 104(A). Computer-readable media may include volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, computer-readable media may include computer storage media and communication media. The graphical interface manager 130 may include additional or alternative components, such as those described below with respect to Figure 6 The components of memory 604 are described.

[0033] The graphical interface manager 130 may be configured to generate a game summary and / or manage the display of the game summary in a user interface (e.g., the user interface 150 of the application 106). The game data capturer 138 may be configured to capture game data, such as metadata, video clips, and screenshots, from the gameplay of the game. The interest determiner 140 may be used by the game data capturer 138 and may be configured to select and / or identify one or more portions of the game data (e.g., game data that may be of interest to the user). In at least one embodiment, one or more portions of the game data capturer 138, the interest determiner 140, and / or the graphical interface manager 130 may be part of the application 106 (and / or the game engine 128).

[0034] like Figure 1 As shown, in one or more embodiments, one or more game summary servers 116 may include at least a portion of a graphical interface manager 130, a game data capturer 138, and / or an interest determiner 140 in addition to a client device 104, or may not include a client device 104. For example, one or more game summary servers 116 may be used in a client-server-based implementation of the GSS system 100, wherein the application 106 comprises a client application. The communication interface 122 may be configured to communicate over any number of networks 102 described herein to receive and / or transmit data used to generate one or more game summaries (e.g., video streams of game play sessions, recordings of game play sessions, audio recordings, video clips, screenshots, user input to one or more input devices 112, streamed content, etc., used to generate metadata and / or generated from data related to gameplay). One or more data repositories 124 may be used to store any of a variety of information associated with the game summary servers 116 (e.g., video clips, screenshots, thumbnails, game summaries, metadata, etc.).

[0035] The game engine 128 of the game server 126 may include gaming functionality that enables one or more users of the client devices 104 to play games over a network. The game engine 128 may be provided at least in part in the game server 126 for a cloud-based implementation of the GSS system 100. In some embodiments, however, the game engine 128 may be included at least in part in the application 106. The communication interface 132 may include one or more components and features for communicating across one or more networks (e.g., one or more networks 102). For example, the communication interface 132 may be used to send and / or receive user input from one or more input devices 112, video data of gameplay (e.g., recorded and / or live streamed), etc.

[0036] As described herein, the applications 106 may include any of a variety of potential types of software capable of presenting one or more game summaries (e.g., game summaries 136 in the user interface 150) on the display 108 of the client device 104. Examples of applications 106 include mobile applications, computer applications, console applications, cloud-based game streaming applications, web browsers, gaming applications, local applications, client applications, social applications, system or native applications, and / or other types of applications or software.

[0037] For cloud-based game streaming applications, the application 106 may include instructions that, when executed by one or more processors, may cause the processors to, without limitation, receive input data representing user input into one or more input devices 112, send the input data to one or more game servers 126, retrieve game data from memory or local storage, receive the game data from the game servers 126 using the communication interface 110, and cause the game to be displayed on the display 108. More generally, in some embodiments, the application 106 may operate as a facilitator to enable interaction between a user and a game instance on a client device 104.

[0038] The application 106 may additionally or alternatively include instructions that, when executed by one or more processors, cause the one or more processors to send data to and receive data (e.g., game data, game summaries, etc.) from one or more game summary servers 116. For example, the application 106 may send video recordings 116 generated while playing a game on one or more client devices 104 to one or more game capture servers 116, and receive video clips, metadata, screenshots, game summary data, and / or other data extracted from the game data from the one or more game capture servers 116. Game data, as used herein, may generally refer to data associated with one or more game play sessions of one or more games, such as video data, audio data, one or more API messages from one or more games (e.g., including or identifying one or more in-game events), and / or user input data (e.g., from a user input device 112). Additionally or alternatively, the one or more game summary servers 116 may receive at least some game data from a game server 126.

[0039] In one or more embodiments, the client device 104 may render the game using the application 106 (e.g., running on the client device 104), while in other examples, the client device 104 may receive display data (e.g., as described in relation to the Figure 8104(B) may receive the encoded display data and display the game on display 108 (e.g., running on one or more game servers 126). In some examples, a first client device (e.g., client device 104(A)) may render the game, while a second client device (e.g., client device 104(B)) may receive the display data and display the game using the display data. In examples where the display data is received by the client device (e.g., where the rendering is not generated by client device 104), GSS system 100 may be part of a game streaming system, such as the following: Figure 8 The game streaming system 800 is described in more detail in .

[0040] Display 108 may include any type of display capable of displaying a game and / or game summary (e.g., a light emitting diode display (LED), an organic LED display (OLED), a liquid crystal display (LCD), an active matrix OLED display (AMOLED), a quantum dot display (QDD), a plasma display, an LED / LCD display, and / or other types of displays). In some examples, display 108 may include multiple displays (e.g., a dual monitor display for computer gaming, a first display for configuring a game, and a virtual reality display for playing the game, etc.). In some examples, the display is a touch screen display, such as a touch screen of a smartphone, tablet computer, laptop computer, etc., where the touch screen is at least one of input devices 112 of client device 104.

[0041] As described herein, the application 106 of the client device 104 can display one or more game summaries, such as the game summary 136 in the user interface 150, via the display 108. To this end, the graphical interface manager 130 can generate one or more game summaries and / or manage the display of the game summaries in the user interface 150 of the application 106. In one or more embodiments, one or more data structures representing the game summaries and / or portions thereof can be generated on the client and / or server side. Rendering the data structures into a user interface (UI) can be handled by an application (e.g., the application 106) running on the client device.

[0042] In at least one embodiment, the graphical interface manager 130 can generate a game summary using metadata of the event log of the game event and corresponding screenshots, video clips and / or other game content, each of which can be automatically generated by the game data capturer 138 based on analyzing the game data associated with the game play session. The metadata can indicate the timing information of the events in the game within one or more game play sessions, as well as the association between the events in the game and the screenshots, video clips and / or other game content that capture the events in the game. As an example and not limitation, the metadata can include timestamps of the events and / or game content in the game, which correspond to the events in the game (e.g., start time, end time, time of non-persistent events, etc.). The timestamp can be used to display data corresponding to the events in the game with a time context (e.g., by a timeline, thumbnails on a grid, icons on a map, per-round statistics, etc.).

[0043] In one or more embodiments, the event log can organize in-game events into rounds or matches of a game play session. For example, metadata can describe a round and a time range or a segment corresponding to a round. Other examples of metadata that can be associated with a time, in-game event, and / or round include in-game object location, score, player status, player statistics, game participants by username, event type, score and score or assist ratio, and / or other information used to configure the game summary 136.

[0044] The game summary 136 displayed in the user interface 150 can take a variety of potential forms and can be displayed in a variety of potential contexts (e.g., using an activity feed, a social feed, a game application, a game streaming application, etc.). By way of example and not limitation, Figure 1 The game summary 136 in FIG. 1 includes a list 155 of in-game events, visual indicators and / or interface elements, a game content display area 142, a timeline 190, a turn display area 192, a game name 152 of the game corresponding to the game summary, and an amount of time 154 of game play in one or more game sessions corresponding to the game summary 136. As described herein, the game summary 136 may include fewer than all of these features and / or different features or combinations thereof.

[0045] List 155 may correspond to screenshots, video clips, and / or other game content captured from a game session (e.g., via game data capturer 138). For example, list 155 includes entries 156, 158, 160, 162, and 164 displayed using visual indicators that may include images (e.g., thumbnails) of the corresponding screenshots and / or video clips. List 155 may take various forms, but is displayed as a ribbon or a carousel. In various embodiments, list 155 may take the form of a carousel, a gallery, a banner rotator, a banner slider, a vertical list, a horizontal list, a grid, and / or a scrolling slider.

[0046] The entries of list 155 may correspond to one or more in-game events of a game play session, and the game content (e.g., video clips and / or screenshots) associated with the entries may capture or otherwise correspond to one or more portions of the corresponding in-game events. Graphical interface manager 130 may use metadata to display the entries in list 155 in an order corresponding to the in-game events that occurred within the game session. For example, timestamps may be used to display the entries in chronological order according to the corresponding in-game events.

[0047] Video clips, screenshots, and / or other game content can capture highlights from a game play session, which the user can view in the game summary 136. For example, the game summary 136 can include interface elements corresponding to in-game events, such as visual indicators and / or associated check boxes, selection areas, and / or buttons (e.g., in or associated with the list 155). The user can select one or more interface elements corresponding to one or more in-game events to cause the corresponding game content to be displayed, played, and / or loaded in the game content display area 142 of the game summary 136.

[0048] For example, when selecting item 158, Figure 1As shown, entry 158 may be emphasized (e.g., highlighted, visually distinguished, etc.) to indicate that an event and / or game content has been selected in one or more corresponding games. Similarly, when selecting entry 158, game content 166 corresponding to the event in the game, such as video clips, screenshots, and / or audio, can be displayed, played, and / or loaded in the game content display area 142 of game summary 136, as shown. When game content 166 includes multiple items, the user can navigate each item using the arrows or other methods shown in game content display area 142. Additionally or alternatively, the arrows and / or other interface elements in game content display area 142 can be used to select another entry in list 155, such as an adjacent entry. When another entry is selected (by list 155, arrows, or other methods), the current entry can be deselected. Deselecting can cause emphasis to be removed from the visual indicator, and corresponding game content is removed and / or replaced with the game content newly selected in game content display area 142.

[0049] When other elements of the game summary 136 correspond to one or more in-game events selected using entries in the list 155, these elements may also be updated based on the selection. For example, other elements may be similarly emphasized and / or de-emphasized upon selection. Examples include entry 168 in the timeline 190 and / or entry 170 in the turn display area 192.

[0050] Based on the metadata, the game summary 136 may include various supplemental information displayed in association with an entry (e.g., in the list 155, timeline 190, turn display area 192, etc.). Supplemental information can help users determine which entries to select. For example, an entry may display an indication of the time and / or duration corresponding to an in-game event within the game session. For example, entry 164 indicates that it corresponds to a 20-second video clip. As a further example, supplemental information may include an indication of the event type of the in-game event associated with the entry. For example, entry 164 indicates that the player, the subject of the game summary 136, scored twice in a row. Other event types may correspond to player scores, assists, wins, losses, upgrades, etc. As a further example, the timeline 190 includes symbols such as a checkmark 178, an "x" 180, and a multiplier 144, which indicate the occurrence of one or more specific event types and / or multiples or variations thereof. Other examples of supplemental information include game state information, such as score information, player information, team information, and / or other information. For example, an entry 170 in the round display area 192 includes score information corresponding to the round (and one or more in-game events). However, game state information can similarly be presented in association with an entry in the list 155 and / or timeline 190.

[0051] A timeline 190 can be displayed to convey the relevant timing of one or more in-game events captured in the event log. Similar to list 155, timeline 190 can be associated with in-game events to indicate the time of the event in the game. The entries in timeline 190 (e.g., entry 168) can be located according to the associated timestamp. The entries corresponding to the in-game events and / or game content automatically extracted by game data capturer 138 can be visually distinguished from the entries manually captured by the player and / or user (e.g., diamonds for manual capture and circles for automatic capture). In at least one embodiment, each entry in timeline 190 can correspond to one or more entries in list 155 (e.g., separately). For example, entry 168 can correspond to the same in-game event as entry 158 in list 155. In one or more embodiments, the entries in timeline 190 can have a one-to-one correspondence with the entries in list 155.

[0052] In one or more embodiments, each entry in the timeline may correspond to one or more game content items similar to the entries in list 155. In addition, one or more entries in timeline 190 may be selectable. For example, upon selecting entry 168, Figure 1 As shown, entry 168 can be emphasized (e.g., highlighted, visually distinguished, etc.), indicating that the corresponding in-game event and / or game content has been selected. Similarly, as shown, when entry 168 is selected, game content 166 (such as, but not limited to, video clips, screenshots, and / or audio) corresponding to the in-game event can be displayed, played, and / or loaded in the game content display area 142 of the game summary 136. If another entry is selected, the current entry can be deselected. Deselection can result in the removal of emphasis from the visual indicator of the entry, and the corresponding game content is removed and / or replaced with the game content newly selected in the game content display area 142.

[0053] Round display area 192 can represent one or more in-game events organized by round. For example, round display area 192 can include entry 170, entry 172, entry 174, and entry 176, which correspond to respective rounds of one or more play sessions of game summary 136. In one or more embodiments, round display area 192 can be used to select a subset of in-game events and / or corresponding entries displayed in one or more other portions of game summary 136. For example, user selection of entry 170 can restrict list 155 and / or timeline 190 to events corresponding to entry 170 (e.g., events occurring within the round) and / or otherwise cause one or more entries to be displayed in those portions of game summary 136 that would otherwise not correspond to entry 170. In one or more embodiments, time on timeline 190 can be scaled according to the time span of the currently selected round. A start time on timeline 190 can correspond to a start time of the round in the play session, and an end time on timeline 190 can correspond to an end time of the round in the play session.

[0054] In one or more embodiments, more than one round can be selected at a time. Also, in one or more embodiments, de-selection of an entry corresponding to a round can remove entries from other portions of game summary 136 that correspond to that round. In some examples, selection of an entry can automatically cause de-selection of one or more currently selected entries. As Figure 1 indicated, selected entries can be emphasized in game summary 136, similar to entries selected in other portions of game summary 136. De-selection of an entry can likewise cause the entry to be deemphasized.

[0055] With respect to Figures 2 to 4 other examples of features that can be included in game summary 136 are described, in addition to those described with respect to Figure 1 game summary 136. Other examples of features that can be included in game summary 136 are described, in addition to those described with respect to Figure 1 game summary 136. Other examples of features that can be included in game summary 136 are described, in addition to those described with respect to Figure 2 , Figure 2 An example of game summary 136 including in-game events of a map 204 of a virtual environment in which one or more in-game events occurred is shown, in accordance with some embodiments of the present disclosure. By way of example and not limitation, Figure 2 game summary 136 includes list 155, game content display area 142, and timeline 190. Game content display area 142 is displayed as including a playback control element, such as a play button and timeline with which a user can interact to view activity occurring during one or more selected in-game events.

[0056] Map 204 can be dynamically updated to illustrate the movement of one or more objects and / or entities over time throughout a gameplay session, as indicated by metadata. For example, the locations of one or more players, NPCs, and / or objects can be superimposed on map 204 or otherwise used to annotate map 204 to indicate where these objects or entities were located within the virtual environment at a particular time in the gameplay session. The time can be based at least on other selections made in game summary 136, such as selected in-game events, game content, and / or entries in user interface 150. For example, if game content display area 142 corresponds to game content 202 including a video clip, in-game events, objects, and / or entities displayed in map 204 can be added, removed, or their locations or other attributes (e.g., health, number of items, score, event type, etc.) can be modified in map 204 as the video clip plays to reflect the game state at the corresponding time in the video clip. The corresponding information used to update map 204 can be captured in metadata, and one or more updates can occur periodically and / or continuously. Interpolation between metadata values ​​can be used to provide intermediate values ​​for the updates. The map 204 may similarly be updated to reflect the game state of other game content in the game content display area 142 , such as a selected screenshot or audio clip.

[0057] In one or more embodiments, the annotated map 204 may include one or more paths that add, remove, and / or update one or more entities or objects throughout the virtual environment. For example, a player's (e.g., the subject of the game summary 136) path 208 throughout the game is shown as a dashed line. The endpoints of the path may correspond to the player's location associated with point 220 in the video clip being viewed. In other examples, the endpoints may correspond to the timestamps of the screenshots being viewed. A view indicator 206 on the map 204 may indicate the player's location in the virtual environment at the current time associated with the game content being viewed, and / or may correspond to an area of ​​the virtual environment that is viewable by the game or otherwise associated with the player at that time. In one or more embodiments, the map 204 may correspond to an in-game map displayed to the player during a game session. The annotations on the map 204 may be limited to information known to the player's account and / or viewable by the player on the in-game map (e.g., at the corresponding time of the game session), or may include information unknown to the player's account and / or viewable by the player (e.g., aggregated from other players or spectators).

[0058] Now refer to Figure 3 , Figure 3 An example of a game summary 136 of in-game events according to some embodiments of the present disclosure is shown, including a game content library 300 that captures one or more in-game events. In one or more embodiments, the library 300 may be about Figure 1 and Figure 2 155 . As described above, the game summary 136 may correspond to in-game events and / or game content from multiple games, across multiple gaming platforms, and / or more than one game play session. For example, a group of game content 302 is indicated as being from game 2, and a group of game content 310 is indicated as being from game 3. As an example and not limitation, an image 304 is indicated to display a map associated with the game session. In one or more embodiments, user selection of the image 304 may display the associated game content in a game content display area 142, which may be included in the game summary 136, added to the game summary 136 based on a selection, and / or included in a pop-up window based on a user selection. In one or more embodiments, a user selection of an entry may result in display of the corresponding map 204, such as with respect to FIG. Figure 2 As stated.

[0059] Now refer to Figure 4 , Figure 4 An example of additional information that may be included in the game summary 136 of an in-game event is shown in accordance with some embodiments of the present disclosure. Figure 4 The game summary 136 includes the game title and indicators 402 of several in-game events captured from the corresponding game play session, along with associated times 406 (e.g., "Today at 1:00 PM"). Below, five images of captured events are shown, such as image 404, which may correspond to the list 155 described herein. As with the entries in the other examples of list 155, tags may be included in images or other visual indicators (e.g., double elimination, score, etc.). These images may serve as visual previews of in-game events and may include thumbnails of basic game content.

[0060] In addition, the game summary 136 includes game state information, such as the level 414 of the game being played (e.g., level 1, 2, 3, etc.), the mode 416 of the game (e.g., team, capture the flag, free-for-all, etc.), and the time the game was played. The game summary 136 may also include a performance or statistical summary 420, which may include performance and / or statistical information, such as the number of eliminations, scores, points, or any other noteworthy events in one or more specific game play sessions or their aggregation. Game statistics 422 may also be included, which may describe important statistics, such as the score per round, how many rounds were played, the number of eliminations, or any other game statistics that may help the user track their progress during the game play session. In aspect, the activity feed may also include an achievement list 424. The achievements 424 will depend on the type of game being played. For example, in a medieval-themed game, the achievements 424 may include the number of goals completed (e.g., towns conquered, enemies eliminated, etc.). In addition to these features, the activity feed may also include a map 428, which shows a map of the game route and the path the user followed throughout the capture event shown in the activity feed summary 400.

[0061] Back to Figure 1 , provides further examples of how to capture in-game events and / or game content from game data. As described herein, game data may refer to data associated with one or more game play sessions of one or more games, such as video data, audio data, one or more API messages from one or more games (e.g., including or identifying one or more in-game events), and / or user input data generated by and / or associated with one or more games (e.g., from user input device 112).

[0062] In one or more embodiments, the application 106 and / or the game data capturer 138 may include instructions that, when executed, record game data from a game play session and store the recorded game data locally on the client device 104 or send the recorded game data to the game summary server 116 or the game server 126 for storage in the data repositories 124 and 134, respectively. In examples where the client device 104 does not generate renderings, the game server 126 may record and store the game data or send the game data to the game summary server 116 for storage in the data repositories 124.

[0063] As described herein, it may be desirable to identify clips (e.g., video and / or audio) and / or other game content from the game data that include interesting content, such as clips in which a greater amount of user input is provided and / or clips in which certain significant events occur during a game play session. Accordingly, a game data capturer 138 may be provided for this purpose.

[0064] As described herein, the game data capturer 138 can be part of the application 106 or can be part of a separate application (e.g., one or more system services, programs, etc.). The game data capturer 138 can include instructions that, when executed by a processor, cause the processor to, without limitation, record or log game data (e.g., input device usage data, video data, and / or other data associated with a game play session). Examples of input device usage data include data describing or representing keyboard, mouse, or other input device usage associated with one or more game play sessions. Examples of information that can be recorded include keyboard strokes, mouse clicks, mouse movements, microphone input, camera input, and / or input to the client device 104 during a game play session. Additionally, the game data capturer 138 can store timestamp information and correlate these inputs with timestamp information for the game play session video data.

[0065] The game data capturer 138 may utilize an interest determiner 140, which may generally be configured to indicate to the game data capturer 138 game content that may be of interest to the user. It is contemplated that the interest determiner 140 may automatically identify game content, or may be manually instructed to do so by the user and / or player (e.g., through one or more user inputs or commands to create a tag, bookmark, capture event, etc.). In one or more embodiments, the interest determiner 140 may analyze game data from a game play session to detect game content. This analysis may be performed on recorded game data, such as after a game session is complete, and / or in real time as the game session occurs. The interest determiner 140 may be part of the application 106, or may be part of a separate application (e.g., one or more system services, programs, etc.). In some examples, the interest determiner 140 is part of the same application as the game data capturer 138.

[0066] Interest determiner 140 can use a variety of potential methods to detect game content and / or corresponding game events in game data (e.g., automatically), examples of which are described herein. To this end, interest determiner 140 analyzes any form of game data, such as game streams, recorded game data, and / or identified game content, to determine and / or detect the occurrence of events in the game. For example, analysis can be used to trigger a capture event and indicate that at least a portion of the game content is captured from the game stream (or saved from a buffer), and / or to classify the detected game event and / or one or more of its attributes (e.g., for inclusion in metadata of one or more game digests 136). Game data capturer 138 can determine that an event has occurred in the game based on artificial intelligence, object detection, computer vision, text recognition, and / or other analytical methods. For example, the interest determiner 140 can utilize any type of machine learning model to detect in-game events and corresponding game content, such as machine learning models using linear regression, logistic regression, decision trees, support vector machines (SVMs), naive Bayes, k-nearest neighbors (KNNs), K-means clustering, random forests, dimensionality reduction algorithms, gradient boosting algorithms, neural networks (e.g., autoencoders, convolutional, recursive, perceptrons, long / short-term memory, Hopfield, Boltzmann, deep belief, deconvolution, generative adversarial, liquid machines, etc.), and / or other types of machine learning models. In some embodiments, the machine learning model includes a deep convolutional neural network.

[0067] Examples of in-game events that can be detected include eliminating another character in the game, collecting an item, scoring, hitting a home run, climbing a tall building or mountain, performing or achieving a user-specified task or goal, leveling up, winning a round, losing a round, and / or other event types. For example, in some embodiments, the interest determiner 140 can identify changes in the crosshairs or other interface elements of the game that indicate an in-game event, such as eliminating a character in the game and triggering a capture event. As a further example, the game data capturer 138 can identify text in the game instance (e.g., using optical character recognition (OCR)) to indicate "Player 1 eliminates Player 4" or "Player 1 scores a touchdown," or other in-game events to trigger one or more capture events. Methods that rely on target detection can analyze game visual data to identify one or more capture events and / or classify game content.

[0068] In one or more embodiments, to identify one or more in-game events, the interest determiner 140 may determine durations of higher levels (e.g., frequency or concentration) of action within a game play session based on the game data. For example, the game data may be used to identify time periods during a game play session that include higher levels of action, and these durations or time periods may be referred to as "durations of predicted interest." When time periods are detected, the disclosed methods (e.g., for screenshots) may be used to identify specific times. For example, the interest determiner 140 may identify time periods within a game play session as high action based at least in part on time periods of high key presses per minute (KPMs), high percentages of action key selections, and / or other input device metrics. Additionally, additional analysis may be performed on these input device metrics to reduce potential noise when identifying time periods of higher action. For example, an interest level algorithm may be applied to the input device metrics to convert the metrics into measurements of running activity over time.

[0069] Based on the value of the interest level algorithm that changes over time, time periods associated with potential high in-game activity can be determined. For example, time periods with data points (e.g., continuously) that are above a threshold value (e.g., an average value) can be identified as being associated with clips or screenshots that may be worth highlighting. The start of a time period can be based on the time when the running activity measurement result exceeds the threshold, and the end of a time period can be based on the time when the running activity measurement result is below the threshold. Video clips or screenshots corresponding to the time period can then be identified. For example, a video clip can be stored as a discrete file that captures frames spanning the time period, and / or can be stored as metadata that is used to identify an area in the game session video data that corresponds to the time period (e.g., using a start timestamp, an end timestamp, and / or a duration).

[0070] While user input (e.g., input device metrics) can be used to quantify activity in a game as an interest level, other types of data can be used in addition to or in place of user input. For example, game session visual and / or audio metrics can be used that correspond to camera and / or field of view movement, color changes, audio volume, audio dynamics, and / or audio changes in the game, etc. Furthermore, while in some examples, the interest determiner 140 can identify a duration of predicted interest and then use that duration to generate a video clip or screenshot, in other examples, the interest determiner 140 can analyze a video clip (generated using any suitable method) to determine whether it is sufficiently interesting, such as by using an interest level algorithm (e.g., based on determining that the average interest level of the clip is greater than a threshold).

[0071] Once a time period is identified as a duration of predicted interest using any of the methods described herein, a corresponding video segment can be identified from the video data of the game play session. For example, the interest determiner 140 can send a timestamp corresponding to the duration of predicted interest to the application 106, the game server 126, and / or the game play capture server 116. Any combination of these components can then use the timestamp to generate and / or identify discrete video clips from the game session video data (e.g., stored in a data repository 124 or 134) and timestamp the video clips or other game content in metadata for one or more game summaries 136.

[0072] The game summary server 116 may include one or more servers for storing, organizing, clustering and / or categorizing game content and / or game data from game sessions. Figure 1 Only some of the components and / or features of the game summary server 138 are shown in FIG. 1 , but this is not intended to be limiting. For example, the game summary server 166 may include additional or alternative components, such as those described below with respect to Figure 9 Components of the computing device 900 are described.

[0073] like Figure 1 As further shown, game summary server 116 may be separate or distinct from game server 126; however, this is not intended to be limiting. In some examples, game summary server 116 may be the same or similar server as game server 126 (e.g., running as a task on game server 126). In some examples, game summary server 116 may be operated or hosted by a first entity (e.g., a first company), while game server 126 may be operated or hosted by a second entity (e.g., a second, different company). In such examples, the second entity may be a game developer, and the first and second entities may share data, such that the first entity may use data received from the second entity to identify interesting game content. In other examples, game play capture server 116 and game server 126 may be operated or hosted by the same entity. In further examples, GSS system 100 may be implemented entirely on client device 104, and / or one or more components and / or their functionality illustrated as included in a server may be implemented at least partially on client device 104.

[0074] The game summary server 116 may include one or more application programming interfaces (APIs) to facilitate communication of information (e.g., game data, timestamps, game content selection data, etc.) with the game server 126 or the client device 104. For example, the game summary server 116 may include one or more game APIs that interface with the client device 104 or the game server 126 to receive game data and / or game summary data. As another example, the game summary server 116 may include one or more APIs that interface with the client device 104 to transmit categorized game content and / or game summary data. Although different APIs are described herein, these APIs may be part of a single API, may combine two or more APIs, may include different APIs than the APIs described herein as examples, or a combination thereof.

[0075] Game servers 126 may include one or more servers (e.g., dedicated game servers) for storing, hosting, managing, and, in some examples, rendering games. In some examples, a first game server 126 may be used to create, update, and modify games (e.g., program code for the games), and a second game server 126 may be used to host games (e.g., as dedicated game servers). Although Figure 1 Only some components and / or features of the game server 126 are shown in FIG. 1 , but this is not intended to be limiting. For example, the game server 126 may include additional or alternative components, such as those described below with respect to Figure 9 Components of the computing device 900 are described.

[0076] The game server 126 may include one or more APIs to enable the client devices 104 to play games and / or to communicate information (e.g., user profiles, game session data, game data, game summaries, etc.) with the game summary server 116 and / or the client devices 104. For example, the game server 126 may include one or more game APIs that interface with the application 106 of the client device 104 to enable the client device 104 to play games. As another example, the game server 126 may include one or more APIs that receive categorized game content and / or other game summary data for transmission to the client devices 104. Although different APIs are described herein, these APIs may be part of a single API, may combine two or more APIs, may include different APIs than the APIs described herein as examples, or a combination thereof.

[0077] The game server 126 may include a game engine 128. The game engine 128 may include functionality that enables one or more users to play a game over a network. The game engine 128 may include a rendering engine, an audio engine, a physics engine, an animation engine, an artificial intelligence engine, a networking engine, a streaming engine, a memory management engine, and / or other components or features. The game engine 128 may be used to generate some or all of the game session data during a game session.

[0078] Now refer to Figure 5 , each block of method 500 and other methods described herein comprises a computing process that can be performed using any combination of hardware, firmware, and / or software. For example, the various functions can be performed by a processor executing instructions stored in a memory. The methods can also be implemented as computer-usable instructions stored on a computer storage medium. The methods can be provided by a standalone application, a service or a hosted service (standalone or in combination with another hosted service), or a plug-in to another product, to name a few. In addition, by way of example, the following are described with respect to Figure 1 However, these methods may additionally or alternatively be performed by any one system or any combination of systems, including but not limited to the systems described herein.

[0079] Figure 5 A flow chart illustrating a method for presenting a game summary including a timeline associated with in-game events, according to some embodiments of the present disclosure, is shown. At block B502, method 500 includes receiving metadata indicating timing information of in-game events. For example, graphical interface manager 130 and / or application 106 may receive metadata indicating the timing of one or more in-game events in one or more game sessions, and an association between the one or more in-game events and one or more video clips capturing the one or more in-game events. In-game events may be determined based at least on analyzing video data representing one or more game play sessions and / or other game data.

[0080] At block B504, method 500 includes using the metadata to present interface elements corresponding to the one or more in-game events and a timeline indicating timing information associated with the one or more in-game events. For example, graphical interface manager 130 and / or application 106 may use the metadata to present one or more interface elements corresponding to the one or more in-game events and a timeline 190 indicating one or more timings associated with the one or more in-game events in user interface 150.

[0081] At block B506, the method 500 includes loading game content that captures the in-game event and updating the timeline to indicate the in-game event based at least on the selection of the interface element. For example, the graphical interface manager 130 and / or the application 106 can load image data of a video clip of one or more video clips in the user interface 150 that captures one or more in-game events and update the timeline to indicate the one or more in-game events based at least on the selection of the interface element of the one or more interface elements that corresponds to the one or more in-game events and based at least on the association between the one or more in-game events and the one or more video clips.

[0082] Referring now to Figure 6 , Figure 6 is a flowchart illustrating a method 600 for presenting a game summary including a list associated with in-game events, in accordance with some embodiments of the present disclosure. At block B602, the method 600 includes receiving metadata indicating times of in-game events. For example, the graphical interface manager 130 and / or the application 106 can access metadata that indicates one or more times of one or more in-game events in one or more play game sessions and includes one or more associations between the one or more in-game events and one or more game content items (e.g., images, videos, etc.) that capture the one or more in-game events.

[0083] At block B604, the method 600 includes presenting, in a list of in-game events using the metadata, an interface element corresponding to an in-game event at a position corresponding to time sequence information of the in-game event in a play game session. For example, the graphical interface manager 130 and / or the application 106 can present, in the list of in-game events using the metadata 155 in the user interface 150, one or more interface elements corresponding to one or more in-game events at one or more positions corresponding to time sequence information of one or more play game sessions.

[0084] At block B606, the method 600 includes presenting game content corresponding to the in-game event based at least on the selection of the interface element. For example, the graphical interface manager 130 and / or the application 106 can present one or more game content items corresponding to the one or more in-game events in the user interface based at least on one or more selections of the one or more interface elements in the list of in-game events and the one or more associations between the one or more in-game events and the one or more game content.

[0085] Referring now to Figure 7 , Figure 7A flow chart of a method 700 for presenting a game summary including a map of a virtual environment associated with an in-game event is shown, according to some embodiments of the present disclosure. At block B702, method 700 includes receiving metadata indicating timing information and a location associated with an in-game event. For example, graphical interface manager 130 and / or application 106 may receive metadata indicating one or more times and one or more locations associated with an in-game event for a game session of a game. The in-game event may be determined based at least on analyzing game data associated with the game session.

[0086] At block B704, method 700 includes using the metadata to render interface elements corresponding to the in-game events and a map of the virtual environment of the game. For example, graphical interface manager 130 and / or application 106 may use the metadata to render one or more interface elements corresponding to one or more in-game events of the game play session and one or more maps of one or more virtual environments of the game.

[0087] At block B706, the method includes annotating the map with the location and timing information based at least on the selection of the interface element. For example, graphical interface manager 130 and / or application 106 may annotate at least one of the one or more maps with one or more location and timing information associated with the one or more in-game events based at least on one or more selections of one or more interface elements corresponding to the one or more in-game events.

[0088] Now refer to Figure 8 , Figure 8 is an example system diagram of a game streaming system 800 according to some embodiments of the present disclosure. Figure 8 Including game server 802 (which may include Figure 1 Game server 126 and / or Figure 9 ), client device 804 (which may include components, features, and / or functionality similar to computing device 900 of Figure 1 The client device 104 and / or Figure 9 900) and network 805 (which may be similar to Figure 1 In some embodiments of the present disclosure, a game streaming system 800 may be implemented. Application sessions may correspond to game streaming applications (e.g., NVIDIA GEFORCE NOW), remote desktop applications, simulation applications (e.g., autonomous or semi-autonomous vehicle simulation), computer-aided design (CAD) applications, virtual reality (VR) and / or augmented reality (AR) streaming applications, deep learning applications, and / or other application types.

[0089] In the game streaming system 800, for a game session, the client device 804 may only receive input data in response to input from an input device, send the input data to the game server 802, receive encoded display data from the game server 802, and display the display data on the display 824. Thus, more computationally intensive calculations and processing are offloaded to the game server 802 (e.g., rendering of the game session is performed by the GPU of the game server 802). In other words, the game session is streamed from the game server 802 to the client device 804, thereby reducing the graphics processing and rendering requirements of the client device 804.

[0090] For example, with respect to instantiation of a game session, the client device 804 can display a frame of the game session on a display 824 based on receiving display data from the game server 802. The client device 804 can receive input from one of the input devices and generate input data in response. The client device 804 can send the input data to the game server 802 via the communication interface 820 and over the network 806 (e.g., the Internet), and the game server 802 can receive the input data via the communication interface 818. The CPU can receive the input data, process the input data, and send the data to the GPU, causing the GPU to generate a rendering of the game session. For example, the input data can represent a user character moving in the game, firing a weapon, reloading, passing a ball, turning a vehicle, etc. The rendering component 812 can render the game session (e.g., representing the results of the input data), and the rendering capture component 814 can capture the rendering of the game session as display data (e.g., capturing image data of a rendered frame of the game session). The encoder 816 can then encode the display data to generate encoded display data, and the encoded display data can be sent to the client device 804 via the network 806 via the communication interface 818. The client device 804 can receive the encoded display data via the communication interface 820, and the decoder 822 can decode the encoded display data to generate display data. The client device 804 can then display the display data via the display 824.

[0091] Figure 9 9 is a block diagram of an example computing device 900 suitable for implementing some embodiments of the present disclosure. The computing device 900 may include an interconnect system or bus 902 that directly or indirectly couples the following devices: memory 904, one or more central processing units (CPUs) 906, one or more graphics processing units (GPUs) 908, communication interfaces 910, input / output (I / O) ports 912, input / output components 914, a power supply 916, one or more presentation components 918 (e.g., a display), and one or more logic units 920.

[0092] In at least one embodiment, computing device 900 may include one or more virtual machines (VMs), and / or any of its components may include virtual components (e.g., virtual hardware components). For non-limiting example, one or more GPUs 908 may include one or more vGPUs, one or more CPUs 906 may include one or more vCPUs, and / or one or more logic units 920 may include one or more virtual logic units. Thus, computing device 900 may include discrete components (e.g., a complete GPU dedicated to computing device 900), virtual components (e.g., a portion of a GPU dedicated to computing device 900), or a combination thereof.

[0093] The interconnection system 902 can represent one or more links or buses, such as an address bus, a data bus, a control bus, or a combination thereof. The interconnection system 902 can include one or more bus or link types, such as an Industry Standard Architecture (ISA) bus, an Extended Industry Standard Architecture (EISA) bus, a Video Electronics Standards Association (VESA) bus, a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. For example, the CPU 906 can be directly connected to the memory 904. In addition, the CPU 906 can be directly connected to the GPU 908. In the case where there is a direct connection or a point-to-point connection between components, the interconnection system 902 can include a PCIe link for performing the connection. In these examples, the computing device 900 does not need to include a PCI bus.

[0094] The interconnect system 902 may represent one or more buses, such as an address bus, a data bus, a control bus, or a combination thereof. The interconnect system 902 may include one or more bus types, such as an Industry Standard Architecture (ISA) bus, an Extended Industry Standard Architecture (EISA) bus, a Video Electronics Standards Association (VESA) bus, a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCIe) bus, and / or another type of bus.

[0095] although Figure 9 The various blocks in FIG90 are shown as being connected to each other via interconnect system 902, but this is not intended to be limiting and is provided for clarity. For example, in some embodiments, a presentation component 918 such as a display device may be considered an I / O component 914 (e.g., if the display is a touch screen). As another example, CPU 906 and / or GPU 908 may include memory (e.g., memory 904 may represent a storage device in addition to the memory of GPU 908, CPU 906, and / or other components). In other words, Figure 9The computing devices referred to in the present disclosure are illustrative only. No distinction is made between terms such as "workstation," "server," "laptop," "desktop," "tablet," "client device," "mobile device," "handheld device," "game console," "electronic control unit (ECU)," "virtual reality system," and / or other device or system types, as all of these are in the Figure 9 The scope of computing devices is contemplated.

[0096] Memory 904 may include any computer-readable medium. Computer-readable media may be any available media that can be accessed by computing device 900. Computer-readable media may include volatile and non-volatile media and removable and non-removable media. By way of example and not limitation, computer-readable media may include computer storage media and communication media.

[0097] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 904 may store computer-readable instructions (e.g., representing programs and / or program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage devices, magnetic cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by the computing device 900. As used herein, computer storage media does not include signals themselves.

[0098] Computer storage media may embody computer-readable instructions, data structures, program modules, and / or other data types in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information transport media. The term "modulated data signal" may refer to a signal that has one or more of its characteristics set or changed in such a manner as to encode information into the signal. By way of example and not limitation, communication media may include wired media such as a wired network or a direct wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Any combination of the above should also be included within the scope of computer-readable media.

[0099] The CPUs 906 can be configured to execute computer-readable instructions in order to control one or more components of the computing device 900 to perform one or more of the methods and / or procedures described herein. Each of the CPUs 906 can include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of handling a large number of software threads simultaneously. The CPUs 906 can include any type of processors and can include different types of processors depending on the type of computing device 900 being implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 900, the processors can be ARM processors implemented using reduced instruction set computing (RISC) or x86 processors implemented using complex instruction set computing (CISC). The computing device 900 can include one or more CPUs 906 in addition to one or more microprocessors or supplemental co-processors such as math co-processors.

[0100] In addition to or in place of the CPU 906, the GPU 908 may also be configured to execute at least some computer-readable instructions to control one or more components of the computing device 900 to perform one or more methods and / or processes described herein. One or more GPUs 908 may be integrated GPUs (e.g., with one or more CPUs 906) and / or one or more GPUs 908 may be discrete GPUs. In embodiments, one or more GPUs 908 may be coprocessors for one or more CPUs 906. The computing device 900 may use the GPU 908 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, the GPU 908 may be used for general-purpose computing on a GPU (GPGPU). The GPU 908 may include hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. The GPU 908 may generate pixel data for outputting an image in response to a rendering command (e.g., a rendering command received from the CPU 906 via a host interface). The GPU 908 may include graphics memory such as display memory for storing pixel data or any other suitable data (e.g., GPGPU data). Display memory can be included as part of memory 904. GPU 908 can include two or more GPUs operating in parallel (e.g., via a link). The link can connect the GPUs directly (e.g., using NVLINK) or through a switch (e.g., using NVSwitch). When combined, each GPU 908 can generate pixel data or GPGPU data for different parts of the output or for different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU can include its own memory or can share memory with other GPUs. In an example where the computing device 900 does not include a GPU 908, a CPU 906 can be used to render graphics.

[0101] In addition to or in lieu of the CPU 906 and / or GPU 908, the logic unit 920 may be configured to execute at least some computer-readable instructions to control one or more components of the computing device 900 to perform one or more methods and / or processes described herein. In embodiments, the CPU 906, GPU 908, and / or the logic unit 920 may perform any combination of methods, processes, and / or portions thereof, either separately or in conjunction. The one or more logic units 920 may be part of and / or integrated within the one or more CPUs 906 and / or the one or more GPUs 908, and / or the one or more logic units 920 may be discrete components of or otherwise external to the CPU 906 and / or GPU 908. In embodiments, the one or more logic units 920 may be processors of the one or more CPUs 906 and / or the one or more GPUs 908.

[0102] Examples of logic unit 920 include one or more processing cores and / or components thereof, such as a data processing unit (DPU), a tensor core (TC), a tensor processing unit (TPU), a pixel vision core (PVC), a vision processing unit (VPU), a graphics processing cluster (GPC), a texture processing cluster (TPC), a streaming multiprocessor (SM), a tree traversal unit (TTU), an artificial intelligence accelerator (AIA), a deep learning accelerator (DLA), an arithmetic logic unit (ALU), an application-specific integrated circuit (ASIC), a floating point unit (FPU), an input / output (I / O) element, a peripheral component interconnect (PCI) or a peripheral component interconnect express (PCIe) element, etc.

[0103] The communication interface 910 may include one or more receivers, transmitters, and / or transceivers that enable the computing device 900 to communicate with other computing devices via an electronic communication network, including wired and / or wireless communications. The communication interface 910 may include components and functionality that enable communication over any of a number of different networks, such as wireless networks (e.g., Wi-Fi, Z-wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communicating over Ethernet or InfiniBand), low-power wide-area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet. In one or more embodiments, the one or more logic units 920 and / or the communication interface 910 may include one or more data processing units (DPUs) for transmitting data received over a network and / or over the interconnect system 902 directly to the one or more GPUs 908 (e.g., their memory).

[0104] The I / O ports 912 can enable the computing device 900 to be logically coupled to other devices including I / O components 914, presentation components 918, and / or other components, some of which can be built into (e.g., integrated into) the computing device 900. Illustrative I / O components 914 include a microphone, a mouse, a keyboard, a joystick, a game pad, a game controller, a satellite dish, a browser, a printer, a wireless device, and the like. The I / O components 914 can provide a natural user interface (NUI) that processes user-generated mid-air gestures, voice, or other physiological input. In some instances, the input can be transmitted to an appropriate network element for further processing. The NUI can implement any combination of voice recognition, stylus recognition, facial recognition, biometric recognition, gesture recognition on and adjacent to the screen, mid-air gestures, head and eye tracking, and touch recognition associated with the display of the computing device 900 (as described in more detail below). The computing device 900 can include a depth camera such as a stereo camera system, an infrared camera system, an RGB camera system, touch screen technology, and combinations thereof for gesture detection and recognition. Additionally, computing device 900 may include an accelerometer or gyroscope to enable motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, the output of the accelerometer or gyroscope may be used by computing device 900 to render immersive augmented or virtual reality.

[0105] The power supply 916 may include a hardwired power supply, a battery power supply, or a combination thereof. The power supply 916 may provide power to the computing device 900 to enable the components of the computing device 900 to operate.

[0106] The presentation component 918 may include a display (e.g., a monitor, a touch screen, a television screen, a head-up display (HUD), other display types, or a combination thereof), speakers, and / or other presentation components. The presentation component 918 may receive data from other components (e.g., the GPU 608, the CPU 906, the DPU, etc.) and output the data (e.g., as images, video, sound, etc.).

[0107] Figure 10 An example data center 1000 is shown, which may be used in at least one embodiment of the present disclosure. The data center 1000 may include a data center infrastructure layer 1010 , a framework layer 1020 , a software layer 1030 , and an application layer 1040 .

[0108] like Figure 10As shown, the data center infrastructure layer 1010 may include a resource coordinator 1012, grouped computing resources 1014, and node computing resources ("node CRs") 1016(1)-1016(N), where "N" represents any complete positive integer. In at least one embodiment, the node CRs 1016(1)-1016(N) may include, but are not limited to, any number of central processing units (CPUs) or other processors (including DPUs, accelerators, field programmable gate arrays (FPGAs), graphics processors or graphics processing units (GPUs), etc.), memory devices (such as dynamic read-only memory), storage devices (such as solid-state drives or disk drives), network input / output (NW I / O) devices, network switches, virtual machines (VMs), power modules and cooling modules, etc. In some embodiments, one or more of the node CRs 1016(1)-1016(N) may correspond to a server having one or more of the above-mentioned computing resources. In addition, in some embodiments, the node CRs

[0109] 1016(1)-1016(N) may include one or more virtual components, such as vGPUs, vCPUs, etc., and / or one or more of the node CRs 1016(1)-1016(N) may correspond to a virtual machine (VM).

[0110] In at least one embodiment, the grouped computing resources 1014 may include separate groups of nodes CR1016 housed in one or more racks (not shown), or many racks (also not shown) housed in data centers at various geographic locations. The separate groups of nodes CR1016 within the grouped computing resources 1014 may include computing, network, memory, or storage resources that can be configured or allocated to support groupings of one or more workloads. In at least one embodiment, several nodes CR1016 including CPUs, GPUs, DPUs, and / or other processors may be grouped in one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0111] Resource coordinator 1012 may configure or otherwise control one or more nodes CR 1016(1)-1016(N) and / or grouped computing resources 1014. In at least one embodiment, resource coordinator 1012 may comprise a software design infrastructure (SDI) management entity for data center 1000. Resource coordinator 1012 may comprise hardware, software, or some combination thereof.

[0112] In at least one embodiment, Figure 10As shown, the framework layer 1020 may include a job scheduler 1032, a configuration manager 1034, a resource manager 1036, and a distributed file system 1038. The framework layer 1020 may include a framework that supports software 1050 of the software layer 1030 and / or one or more applications 1042 of the application layer 1040. The software 1050 or the application 1042 may include web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. The framework layer 1020 may be, but is not limited to, a free and open source software network application framework, such as Apache Spark, which can utilize the distributed file system 1038 for large-scale data processing (e.g., "big data"). TM (hereinafter referred to as "Spark"). In at least one embodiment, the job scheduler 1032 may include a Spark driver to facilitate scheduling of workloads supported by the various layers of the data center 1000. In at least one embodiment, the configuration manager 1034 may be capable of configuring the various layers, such as the software layer 1030 and the framework layer 1020 including Spark and a distributed file system 1038 for supporting large-scale data processing. The resource manager 1036 may be capable of managing the cluster or group computing resources mapped to or allocated to support the distributed file system 1038 and the job scheduler 1032. In at least one embodiment, the cluster or group computing resources may include the grouped computing resources 1014 at the data center infrastructure layer 1010. The resource manager 1036 may coordinate with the resource coordinator 1012 to manage these mapped or allocated computing resources.

[0113] In at least one embodiment, the software 1050 included in the software layer 1030 may include software used by at least a portion of the node CRs 1016(1)-1016(N), the grouped computing resources 1014, and / or the distributed file system 1038 of the framework layer 1020. The one or more types of software may include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0114] In at least one embodiment, the one or more applications 1042 included in the application layer 1040 may include one or more types of applications used by at least a portion of the node CRs 1016(1)-1016(N), the grouped computing resources 1014, and / or the distributed file system 1038 of the framework layer 1020. The one or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in conjunction with one or more embodiments.

[0115] In at least one embodiment, any of the configuration manager 1034, resource manager 1036, and resource coordinator 1012 can implement any number and type of self-modification actions based on any number and type of data acquired in any technically feasible manner. The self-modification actions can relieve the data center operator of the data center 1000 from making potentially poor configuration decisions and can avoid underutilized and / or poorly performing portions of the data center.

[0116] The data center 1000 may include tools, services, software, or other resources for training one or more machine learning models or using one or more machine learning models to predict or infer information according to one or more embodiments described herein. For example, a machine learning model may be trained by calculating weight parameters according to a neural network architecture using the software and computing resources described above with respect to the data center 1000. In at least one embodiment, using the weight parameters calculated using one or more training techniques, the resources described above with respect to the data center 1000 may be used to infer or predict information using a trained machine learning model corresponding to one or more neural networks, such as but not limited to those described herein.

[0117] In at least one embodiment, data center 1000 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or reasoning using the aforementioned resources. In addition, one or more of the aforementioned software and / or hardware resources may be configured as a service to allow users to train or perform information reasoning, such as image recognition, speech recognition, or other artificial intelligence services.

[0118] Sample network environment

[0119] A network environment suitable for implementing embodiments of the present disclosure may include one or more client devices, servers, network attached storage (NAS), other backend devices, and / or other device types. The client devices, servers, and / or other device types (e.g., each device) may be configured to: Figure 9 The backend device may be implemented on one or more instances of the computing device 900 of the data center 1000—for example, each device may include similar components, features, and / or functionality of the computing device 900. In addition, in the case of implementing a backend device (e.g., a server, NAS, etc.), the backend device may be included as part of the data center 1000, examples of which are described herein with respect to Figure 10 Describe in more detail.

[0120] The components of the network environment can communicate with each other through the network, which can be wired, wireless, or both. The network can include multiple networks, or a network of networks. For example, the network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks (e.g., the Internet and / or the Public Switched Telephone Network (PSTN)), and / or one or more private networks. In the case where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connections.

[0121] Compatible network environments may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment), and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein with respect to the server may be implemented on any number of client devices.

[0122] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, and the like. The cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework for supporting software at the software layer and / or one or more applications at the application layer. The software or application may include network-based service software or application programs, respectively. In an embodiment, one or more client devices may use network-based service software or application programs (e.g., by accessing the service software and / or application programs via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, a type of free and open source software network application framework that may, for example, use a distributed file system for large-scale data processing (e.g., "big data").

[0123] A cloud-based network environment can provide cloud computing and / or cloud storage that performs any combination of the computing and / or data storage functions described herein (or one or more portions thereof). Any of these various functions can be distributed across multiple locations from a central or core server (e.g., one or more data centers that can be distributed across a state, region, country, global, etc.). If the connection to the user (e.g., client device) is relatively close to an edge server, the core server can assign at least a portion of the functionality to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0124] Client devices may include Figure 9 The client device 900 may be embodied as a personal computer (PC), a laptop computer, a mobile device, a smartphone, a tablet computer, a smartwatch, a wearable computer, a personal digital assistant (PDA), an MP3 player, a virtual reality head-mounted display, a global positioning system (GPS) or device, a video player, a camera, a surveillance device or system, a vehicle, a watercraft, an aircraft, a virtual machine, a drone, a robot, a handheld communication device, a hospital device, a gaming device or system, an entertainment system, an in-vehicle computer system, an embedded system controller, a remote control, an appliance, a consumer electronic device, a workstation, an edge device, any combination of these described devices, or any other suitable device.

[0125] The present disclosure can be described in the general context of machine-usable instructions or computer code executed by a computer or other machine such as a personal digital assistant or other handheld device, including computer-executable instructions such as program modules. Generally, program modules including routines, programs, objects, components, data structures, etc. refer to code that performs a specific task or implements a specific abstract data type. The present disclosure can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. The present disclosure can also be practiced in a distributed computing environment where tasks are performed by remote processing devices linked through a communication network.

[0126] As used herein, the statement "and / or" with respect to two or more elements should be interpreted as referring to only one element or combination of elements. For example, "element A, element B and / or element C" may include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or elements A, B and C. In addition, "at least one of element A or element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, "at least one of element A and element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0127] The subject matter of the present disclosure is described in detail herein to meet statutory requirements. However, the description itself is not intended to limit the scope of the present disclosure. On the contrary, the inventors have contemplated that the claimed subject matter may also be embodied in other ways to include steps that are different from the steps described herein in conjunction with other current or future technologies, or combinations of similar steps. Moreover, although the terms "step" and / or "block" may be used herein to imply different elements of the method employed, these terms should not be interpreted as implying any particular order among or between the various steps disclosed herein, unless the order of the steps is explicitly described.

Claims

1. A method for generating a game summary using metadata, comprising: receiving metadata indicating timing information of an in-game event within a game play session, a first association between the in-game event and a video clip capturing the in-game event, and a second association between the in-game event and a round of a plurality of rounds of the game play session, wherein the in-game event is determined based at least on analyzing video data representing the game play session; presenting, in a graphical user interface and using the metadata, interface elements corresponding to the events in the game and a timeline indicating the timing information associated with the events in the game; displaying, in the graphical user interface, image data of a video clip of the video clip that captures the in-game event based at least on a selection of the interface element of the interface element that corresponds to the in-game event and the first association; as well as The selection causes a timeline of in-game events displayed in the graphical user interface to be updated to correspond to the turn.

2. The method according to claim 1, wherein The selection includes at least images corresponding to the video clip and the in-game event.

3. The method according to claim 1, wherein The selection is from an in-game event list that uses the metadata to display a plurality of the in-game events in an order corresponding to the times of the in-game events.

4. The method according to claim 1, wherein The selection further results in an update to a round display area of ​​the graphical user interface.

5. The method according to claim 1, wherein The updating includes an indicator that visually emphasizes events in the game in the timeline.

6. The method according to claim 1, wherein The metadata further associates the in-game event with an event type, the selection being from a list of in-game events, and the metadata is used to display the event type with the corresponding in-game event in the list of in-game events.

7. The method according to claim 1, wherein The updating further includes annotating a map of a virtual environment in which the in-game event occurs with the metadata based at least on one or more instances of the timing information associated with the in-game event.

8. A processor comprising: one or more circuits for accessing metadata indicating timing information corresponding to an in-game event within a game play session, including an association between the in-game event and one or more images capturing the in-game event, and associating the in-game event with a turn in a plurality of turns of the game play session, Using the metadata, presenting, in a list of in-game events in a user interface, an interface element corresponding to the in-game event at a position corresponding to the timing information within the game play session; presenting, in the user interface, the one or more images corresponding to the in-game event based at least on a selection of the interface element from the in-game event list and the association between the in-game event and the one or more images, and The selection causes a timeline of in-game events displayed in the user interface to be updated to correspond to the turn.

9. The processor according to claim 8, wherein: The one or more images include a video clip of the game play session.

10. The processor of claim 8, further comprising: Based at least on the selection, the interface element of the in-game event is visually emphasized in the in-game event listing.

11. The processor according to claim 8, wherein: The metadata further associates the in-game event with an event type, and the metadata is used to display the interface element of the in-game event together with an indicator of the event type.

12. The processor according to claim 8, wherein: The selection further results in an update to a turn display area of ​​the user interface.

13. The processor of claim 8, further comprising: Based at least on the selection, game state information displayed in the user interface is updated using the metadata to correspond to the in-game event.

14. The processor of claim 8, further comprising: A map of a virtual environment in which the in-game event occurs is annotated with the metadata based at least on the timing information corresponding to the in-game event.

15. A system for generating a game summary using metadata, comprising: one or more processing units; One or more memory devices storing instructions that, when executed using the one or more processing units, cause the one or more processing units to perform a method comprising: receiving metadata indicating timing information and an in-game position associated with an in-game event of a game play session of a game, wherein the in-game event is determined based at least on analyzing game data associated with the game play session; Using the metadata, presenting interface elements corresponding to in-game events of the game play session along with a map of a virtual environment of the game; and The map is annotated with the location and the timing information associated with the in-game event based at least on a selection of one or more of the interface elements corresponding to the in-game event.

16. The system according to claim 15, wherein: The annotation includes presenting a path of an in-game object in the game on the map, wherein endpoints of the path are based on at least the timing information.

17. The system according to claim 15, wherein: The annotation includes presenting a location of an in-game object of the game on the map based on at least the location and the timing information.

18. The system according to claim 15, wherein: The annotation includes presenting, on the map, locations of one or more in-game events of the game based at least on the locations and the timing information.

19. The system of claim 15, wherein: The annotation includes presenting a symbol for the in-game event based at least on an event type specified for the in-game event in the metadata.

20. The system of claim 15, wherein: The interface element is displayed in a timeline of a user interface at a location corresponding to timing information corresponding to the in-game event in the game play session.

Images