AI-generated ghost player

Abstract

Methods and systems are provided for providing assistance for players during game amusement of a video game. The method includes executing a video game for a game session. The player controls a player character during the game session. The method includes detecting a selection of ghost assistance for the game session. The method includes linking or connecting the game session with an Assisted Artificial Intelligence (AI) engine. The assist AI engine operates to provide ghost assist to the player according to a mode setting. The ghost assistance is provided via a ghost role. The method includes rendering the ghost role during the game session to perform an interactive action in the game. The interactive action is generated in response to a control input generated by the secondary AI engine. The interactive action of the ghost role is configured to cause the ghost role to progress along an interactive path of the game.

Description

Technical Field

[0001] This disclosure generally relates to computer-implemented methods for providing ghost assistance that is customized based on the player's current game play and mode settings. Background Technology

[0002] Description of related technologies The video game industry has undergone many changes over the years. Users are now able to play video games using a wide variety of peripherals and computing devices. Sometimes video games are played using game consoles, where the console handles the game and generates interactive input displayed on the screen. In other cases, video games are played in a streaming mode, where one or more servers remotely execute the game, and the user provides input via a network-connected device.

[0003] Despite advancements in video game technology, some players still find themselves needing assistance. Games have become increasingly complex, often causing non-expert players to give up or find tasks too difficult to complete. Players can research games or even search for previous gameplay on the internet, but this process is time-consuming and often irrelevant to the player's current task and / or scenario.

[0004] To help address this issue, some solutions have included using avatars that users can follow. However, these avatars are typically graphical or outlined images of previous players who have played the game, and the user must follow the avatar, observing the actions taken (including actions and interactions unrelated to the user's current gameplay context). This is not very helpful for providing real-time assistance to players who are having difficulty with specific game scenarios or following a particular gameplay path. This is because the avatar is unaware of what specific game the player is playing or the context of their gameplay. Furthermore, the avatar often only performs actions from a previous player. Even further, the avatar is presented only within a general game context relevant to the game itself, rather than the player's current gameplay context.

[0005] Against this backdrop, the implementation method of this disclosure is proposed. Summary of the Invention

[0006] Implementations of this disclosure include methods, systems, and apparatus for providing ghost assistance to a player during a game session, wherein the game assistance is intelligently provided by a ghost character based on control inputs learned by an artificial intelligence (AI) model of the game and selected mode settings therefrom, the model being trained from multiple sources of training material. Generally, the AI ​​model is trained using training material from the game's gameplay to identify game scenarios and learn the interactions required to progress in the game from the training material. During live gameplay, game state data is analyzed to identify scenarios appearing in the player's gameplay context. These scenarios can then be processed by the AI ​​model and an auxiliary AI engine to provide the ghost character with control inputs for animation / control, guiding the player in relevant game interactions. These relevant interactions are context-specific to the player's gameplay context, not to another player's previous gameplay. Therefore, the ghost character can be controlled by the auxiliary AI engine during gameplay and provide the player with a visual representation of how to play certain game scenarios, enabling the player-controlled character to progress in the game.

[0007] In some implementations, the way the auxiliary AI engine controls the ghost character 113 is used to identify pattern settings. Sometimes, the ghost character provides limited assistance; sometimes, it provides detailed and comprehensive assistance; and sometimes, it provides assistance along a reduced narrative of the game, focusing on specific features or playstyles. Thus, the AI-generated ghost player can act as a guide, providing an overlay for the player showing how to perform gameplay to achieve progress. In some implementations, the ghost character may also provide sample controller input button sequences. In one implementation, the ghost character can maintain interactive dialogue with the player character (e.g., providing guidance or instructions related to the game).

[0008] In one implementation, the overlay takes the form of a ghost character. The ghost character can be an outline character, a shadow character, or a fully animated character (i.e., not in ghost form). In some implementations, the ghost character can interact with the player. In some implementations, the player can seek assistance from the ghost character regarding natural language queries. In other implementations, the ghost character can be represented as a character from a movie, a character from another game, or a user-generated character. In yet another implementation, the ghost character can be represented as a custom character created by the player. Generally, the ghost character provides assistance because it achieves this by learning from game scenarios and interactions derived from training materials (obtained from multiple different sources).

[0009] In one embodiment, a method is provided for providing assistance to a player during gameplay of a video game. The method includes executing the video game for a game session. The player controls a player character during the game session. The method includes detecting the selection of a ghost assistance for the game session. The method includes linking or connecting the game session to an assistive artificial intelligence (AI) engine. The assistive AI engine operates to provide ghost assistance to the player based on mode settings. The ghost assistance is provided via a ghost character. The method includes rendering an interactive action performed by the ghost character in the game during the game session. The interactive action is generated in response to control input generated by the assistive AI engine. The interactive action of the ghost character is configured to allow the ghost character to progress along an interactive path in the game. The ghost character assists the player interacting with the player character along the interactive path. The interactive action of the ghost character is visible to the player, thereby providing the ghost assistance.

[0010] In some implementations, the AI ​​engine uses an AI model trained with game-related playable content.

[0011] In some implementations, game entertainment materials are processed to extract contextual features, which are then used by AI models for classification so that the AI ​​models can recognize game scenes.

[0012] In some implementations, control inputs generated by an auxiliary AI engine are used to control the ghost character to perform interactive actions on one or more game scenes in a game session in order to make progress in the game.

[0013] In some implementations, the AI ​​engine uses an AI model trained with game play assets associated with previous gameplay to identify game scenes, which are used by the auxiliary AI engine to generate control inputs for enabling the ghost character to make progress in game scenes that appear during the player's current game session.

[0014] In some implementations, the mode setting defines the type of assistance provided by the ghost character during ghost assistance.

[0015] In some implementations, the mode setting is configured to switch during a game session.

[0016] In some implementations, switching to a different mode setting changes the type of assistance provided by the ghost character. The mode setting is selected from Story Mode, Battle Mode, Exploration Mode, and Full Game Mode. Other modes can be defined based on one or more new or custom narratives selected from options or provided / defined by the user.

[0017] In some implementations, the AI ​​engine uses an AI model trained with game play assets associated with previous gameplay of the game to identify game scenes, which are used by the auxiliary AI engine to generate control inputs for enabling the ghost character to progress in game scenes appearing during the player's current game session, and wherein the control inputs change when a mode setting switch is detected.

[0018] In some implementations, mode settings switching allows the ghost character to focus on different gameplay scenarios that provide ghost assistance to the player.

[0019] In some implementations, Ghost Assist is a type of guided mode that guides the player character along an interactive path.

[0020] In some implementations, Ghost Assist is a type of Full Mode in which a Ghost character completes one or more interactive tasks for the player character along an interactive path.

[0021] In some implementations, game assets are selected from one or more of the following: user-generated content (UGC) game entertainment assets; assets containing game narrative content; assets containing game commission content; assets containing achievement content; assets containing character type content; assets containing storyline content; assets containing in-game skins; assets containing game objects; game assets captured within or outside the network; assets containing user settings data; assets containing privacy control data; assets containing age settings; or a combination of two or more of the above.

[0022] Other aspects and advantages of this disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrates the principles of this disclosure by way of example. Attached Figure Description

[0023] This disclosure can be better understood with reference to the following description taken in conjunction with the accompanying drawings, in which: Figure 1 Flowchart 100 is shown, illustrating example method operations performed according to one implementation to provide ghost assistance to a player during gameplay.

[0024] Figure 2 A flowchart process for providing ghost assistance is shown according to one implementation.

[0025] Figure 3 Another flowchart is provided, illustrating another example of the provision of ghost assistance according to one implementation scheme.

[0026] Figure 4AA flowchart is shown illustrating the activation of Ghost Assist during gameplay according to one implementation scheme.

[0027] Figure 4B Another example is shown that provides a mode setting switch for controlling the operation of the auxiliary AI engine.

[0028] Figure 5 Examples of apparatus and / or platform system components that can be used to carry out various embodiments of this disclosure are shown. Detailed Implementation

[0029] The following implementations of this disclosure provide methods, systems, and apparatus for providing ghost assistance to a player during a game session, wherein the game assistance is intelligently provided by a ghost character based on a model of game play from multiple training material sources and a mode setting selected therefrom. Generally, an AI model is trained using training material from the game play to identify game scenarios and learn the interactions required to progress in the game from the training material. During live game play, game state data is analyzed to identify scenarios that occur within the player's game play context. These scenarios can then be processed by the AI ​​model and an auxiliary AI engine to provide control input to the ghost character for guiding the player in aspects of game interaction. Therefore, the ghost character can be controlled by the AI ​​auxiliary engine during game play and provides the player with a visual representation of how to play certain game scenarios so that the player-controlled character can progress in the game.

[0030] In light of the above overview, several example figures are provided below to facilitate understanding of the example implementations.

[0031] Figure 1 Flowchart 100 illustrates an example method of operation performed according to one implementation to provide ghost assistance to a player during gameplay. As shown, game engine 102 is configured to execute game 104. Game 104 can be a game executed locally on a game console or computer, or a streaming game executed in the cloud. Game 104 is a game created based on game logic and a game engine to enable gameplay. Users with access to the game can select game 104 to play, whether in the first session of gameplay or in subsequent game sessions. Typically, gameplay lasts for a period of time, during which the player progresses through one or more sessions of gameplay.

[0032] In one implementation, game 104's game entertainment 112 is shown in Figure 1This general example illustrates a player character 109 (which can be controlled by a player 107) rendered for game play 112 by game 104 and game engine 102, along with game objects and scenes. Player 107 is shown interacting with controller 103 and providing control input to drive the interactions of player character 109 during game play 112. In other embodiments, keyboards, joysticks, gestures, and other interactive inputs may be used to drive the interactions of player character 109 during game play 112.

[0033] In this example, a ghost character 113 is rendered in game play 112 during a game session. The ghost character 113 is rendered to provide ghost assistance to player 107, which enables player character 109 to follow or learn what interactions should be performed to progress in the game. As mentioned above, the ghost character 113 can be a character's outline, a character's shadow, another character, a fully animated character, a partially animated character, a custom character, a character rendered with the look and feel of a cinematic character, a stick figure, an animated head or face, animated hands and parts of a character's body, or a combination of two or more of the above.

[0034] Progression in a game can include moving from one game scene to another, acquiring interactive skills, learning input combinations, learning new routes to take throughout the game world, overcoming obstacles, defeating bosses, scoring points, winning trophies, or generally progressing from one stage to another or from one level to another.

[0035] In this embodiment, Ghost Assist 106 can be selected by player 107 to play the game with the assistance provided by Ghost Character 113. In one embodiment, Ghost Assist 106 is a function, code, subsystem, plugin, middleware, or a combination thereof provided by game 104. In another embodiment, Ghost Assist 106 may be provided by system software, such as the system software of a game console or system software enabled by a personal computer or other computing device. In either embodiment, Ghost Assist 106 can be selected by player 107 before or during gameplay.

[0036] In addition to selecting Ghost Assist 106, mode setting 116, such as that selected by player 107, is also recognized. In other implementations, the interaction logic may select or suggest mode setting 116 for player 107 to choose, or mode setting 116 may be applied automatically by the system, code, or artificial intelligence engine.

[0037] Input mode setting 116 assists the artificial intelligence (AI) engine 108. The assist AI engine 108 will receive the mode setting 116 selection and use this setting to activate the type of assistance it will provide to the player 107 via the ghost character 113. As will be described below, the type of assistance may vary depending on the mode setting 116. In one implementation, an artificial intelligence (AI) model 110 is generated, trained, and provided as an agent for the assist AI engine 108.

[0038] For example, the AI ​​model is trained over time using various types of training material from previous game positions in game 104. Training material may include gameplay from previous sessions by player 107 or multiple other players accessing the game via a network, service, online server, or cloud-based system. In some implementations, an application programming interface (API) may be used to access one or more websites that may contain media (e.g., images, videos, clips, shared sequences, posted screenshots, mini-games, etc.) associated with one or more of the aforementioned types of training material. Utilizing the various types of training material, AI model 110 is configured to extract contextual features from the diverse training material to classify the interactions contained within it. The extracted features will assist the classification used by the AI ​​model to identify game scenarios. The game scenarios identified from the training material can then be learned by the AI ​​model to detect interactive inputs that will contribute to achieving specific outcomes in different game scenarios.

[0039] For example, control inputs can be generated by the auxiliary AI engine 108 in response to a game scene identified by the AI ​​model 110, and then used to provide those control inputs to drive the interactions of the ghost character 113. Thus, the ghost character 113 will interact in the game 112 based on the interaction inputs generated by the auxiliary AI engine 108, which are rendered for the player 107 to view while controlling the player character 109. In one implementation, the AI ​​model 110 is trained using multiple training materials from a number of users who have played the game 104. Previous game play can come from a variety of sources, including a database of previous game play performed using a play network, executed and recorded game play (two videos rendered on a third-party video streaming service), short game play videos posted to social networks, video clips, game play videos and descriptive reviews obtained from internet websites, or a combination thereof.

[0040] Additionally, during gameplay 112, which is executed to render a session for player 107, game state 114 generated during gameplay 112 is provided to AI model 110 as a feed channel. Game state 114 is used by AI model 110 to identify game scenarios from previous gameplay that may be similar to game scenarios appearing in real-time during gameplay 112. For example, a game scenario appearing in gameplay 112 might show a bicycle that can be used in the game. Game state 114 identifies this game scenario, which is then feature-processed and classified by AI model 110. Furthermore, one or more other training materials may have been previously processed to identify scenarios or similar scenarios featuring bicycles and game objectives. The auxiliary AI engine can then generate control input for a ghost character 113, which will show player 107 how to ride or use the bicycle to progress, score points, or complete one or more tasks in the game.

[0041] Therefore, once a game scenario is identified for the current gaming session, the assistive AI engine 108 can utilize information from the training materials to select the control inputs to drive the interactions of the ghost character 113. In this way, the ghost character 113 will provide real-time guidance and instruction to the player 107, allowing the player character 109, controlled by the player 107, to attempt to follow and repeat these interactive actions and inputs. As will be described in more detail below, pattern settings serve as a type of filter that defines the type of assistance the ghost character 113 will provide. In general, assistance can range from very detailed or comprehensive to more general or intermittent. Besides limiting the amount of assistance, pattern settings can also be used to modify the nature of the assistance. Thus, the assistive AI engine 108 can provide the ghost character 113 with control inputs consistent with pattern settings defined by the user or recommended based on the user's profile, prior use, or skills.

[0042] Figure 2A flowchart illustrating a process for providing ghost assistance according to one embodiment is shown. In this example, a game session is executed for a game in operation 202. The game session can be executed on a local machine or on a remote computer such as a server. The remote machine can be executed in a data center where one or more games (e.g., game titles) are accessible to players with accounts on a cloud system. Thus, gameplay is processed and executed on the server and streamed to a remote player who can be playing the game on any type of device connected to the internet. In the local computer example, the player can be playing the game on a personal computer, game console, or handheld device. In some embodiments, the local computer may include a head-mounted display (HMD). In one configuration, the selection of ghost assistance in operation 204 can be detected before the game session is operational (e.g., before playing the game) or at any time during gameplay. This provides a functional feature that can be selected by the player upon request during or in preparation for gameplay. Thus, all or part of the game session can include gameplay assisted by utilizing ghost assistance functionality.

[0043] In operation 206, the game session is linked to an assistive artificial intelligence (AI) engine operating in a specific game mode. Generally, the process of linking the assistive AI engine can be a result of the player choosing to use the ghost assist (e.g., through drop-down menus on a graphical user interface, button presses, verbal input, gestures, or combinations thereof). Linking should be broadly understood to include simply activating, connecting to, or using the ghost assist function or functionality. In some embodiments, the ghost assist functionality can be executed by a third-party server, where an application programming interface (API) can be used. In another embodiment, the ghost assist functionality can be integrated as part of the game logic of the game. In yet another embodiment, the ghost assist functionality can be integrated as part of system logic (e.g., system logic used by an entertainment system or personal computer). Furthermore, game assist functionality can also be processed by the game console operating system and made functionally operable in response to user selection or system selection (via a recommended or pre-planned operating mode of the assist functionality).

[0044] In operation 208, a ghost character is rendered to perform interactive actions within the game's gameplay based on control input. The control input is generated by an auxiliary AI engine to guide the player along the game's interactive path. Generally, the ghost character can be generated as a transparent game character that can display game interactions that are viewable and visible to the player controlling the player character during the gameplay session. Therefore, the ghost character 113 can perform actions, movements, inputs, and tricks, which can be followed by the player character 109 in response to controls from the player 107. In one embodiment, the actions performed by the ghost character 113 can be viewed by the player 107, but in some embodiments, this may not advance actual interactive gameplay.

[0045] For example, if Ghost Character 113 scores a touchdown, the score associated with that touchdown may not be counted in the game. For instance, Ghost Character 113 could be shown as having scored a touchdown, and Player 107 could subsequently attempt to score a touchdown in the same way, but the score achieved by Ghost Character 113 would not be counted, or a replay of the actual action performed by Ghost Character 113 would be canceled. This links game interaction and progress to actions performed by Player Character 109, not Ghost Character 113.

[0046] In another implementation, the ghost character 113 can perform actions that actually score points and make them effective to advance the player (i.e., to contribute to the player 107's progress in the game). When the ghost character 113 scores, advances, or gains points, the player character 109 can simply follow the ghost character 113 and attribute such points to the player character 109. In some other implementations, a hybrid approach can be adopted, where some actions performed by the ghost character 113 are credited to the player, and some actions must be performed by the player character 113 to credit points to the player. In some cases, this hybrid approach can be periodically activated and deactivated during gameplay (e.g., based on player input or automatically via an algorithm).

[0047] This mode can be associated with the full mode. The full mode can be selected as one of the mode settings in mode setting 116. In full mode, the ghost character 113 actually completes game tasks and advances the player. In contrast, the guided mode shows guided examples of interactions performed by the ghost character 113, which the player character 109 must follow and complete to advance. In both cases, the ghost character is rendered within a scene of gameplay performed by the player during a game session.

[0048] Figure 3Another flowchart is provided, illustrating another example associated with providing ghost assistance according to one implementation. In operation 300, it is confirmed that the ghost assistance is provided by game logic or system logic. As mentioned above, the game can integrate accessibility functionality and auxiliary AI engine code for use during gameplay. In another implementation, the ghost assistance logic and the associated auxiliary AI engine can be executed and managed by game system logic. In one implementation, the game system logic can be the system logic for a console used for gameplay. In another implementation, the system logic can be stored in the cloud as part of a service and executed in the cloud, for example, for streaming the game.

[0049] In operation 302, an action card is used to identify the ghost assist selection input. For example, the ghost assist logic may include a settings tree that can be organized to provide input for selecting different types of modes. As shown, once the ghost assist selection input has been selected in operation 302, mode setting option 304 can be provided. In one implementation, the user can select either a guided mode 306 or a full mode 308. As mentioned above, guided mode 306 allows the user to follow a ghost player, generated by the computer from a collection of game play videos across the entire game world, as a guide through the game. Full mode 308 allows the user to temporarily relinquish control to a ghost character and watch the ghost character complete levels for the user, thereby relieving user stress.

[0050] In some implementations, mode setting 304 may also include more detailed mode settings that define the style of the ghost assistance. For example, mode settings associated with different styles may include story mode 310, battle mode 312, exploration mode 314, full game mode 316, etc. Based on the selection, a ghost character selection 318 is performed, the ghost character being the one whose behavior will be performed according to the selected mode setting 304.

[0051] In one example, Story Mode 310 could include a ghost narrative with an auxiliary AI engine that compiles only the necessary story fragments, allowing the player to follow the ghost through the game and complete the storyline. In Combat Mode 312, the auxiliary AI engine would utilize combat movement from the game to compile the ghost narrative, allowing the player to follow the ghost while focusing on combat scenarios. In Exploration Mode 314, a completion list type mode would be implemented, where the auxiliary AI engine would compile a ghost narrative focusing on discovery moments and the collection of new and potentially rare items, weapons, new areas, etc. In Full Game Mode 316, the auxiliary AI engine would allow the AI ​​ghost to guide the user through the game, just as in a typical sequential gameplay experience. It is understood that such choices for Mode Setting 304 would allow players to choose between different types and methods (styles) of ghost character assistance. Generally, the ghost character generated by the ghost assistance logic serves as a visual overlay of AI-generated gameplay assets, helping to guide the user throughout the game.

[0052] AI characters (e.g., ghost characters) are created within an aggregate of user-generated content (UGC) from in-game moments across all systems (including internal web content, third-party videos, social media sites, video media sites, game media sites, etc.). All of this information can be used as... Figure 1 The training materials shown are (TF1 to TFn). In this way, AI model 110 continuously learns from various game videos and game scenarios that typically appear in game content, in order to learn how to play games in various scenarios. Then, auxiliary AI engine 108 can access AI model 110 according to mode settings 116 to provide the necessary input for guiding and controlling the ghost character 113 in the player 107's game.

[0053] continue Figure 3Once mode setting 304 is defined, mode setting switching 320 can define the logic that causes the auxiliary AI engine 108 to change its behavior during gameplay 112. For example, a player can switch between story mode 310 and exploration mode 314 midway through a session. In other embodiments, switching may include changes between operations under guided mode 306 or full mode 308. These changes in mode setting switching 320 can occur instantaneously (e.g., during gameplay or during pauses, or before a specific level or quest, or a combination thereof). Therefore, the game auxiliary AI engine 108 dynamically provides control input based on the mode setting 304 selected and / or switched to during a game session to guide the ghost character 113 in real time within gameplay 112. In some embodiments, mode setting switching 320 can be dynamically performed by the system based on learned training. For example, machine learning can be used to identify when players prefer a particular mode setting to complete a specific task, which in turn causes the mode setting 314 to change dynamically, allowing the auxiliary AI engine 108 to behave differently depending on the game context and the scenario encountered by the player.

[0054] Figure 4A A flowchart is shown illustrating the activation of Ghost Assist during game play 402 according to one implementation. As shown, during operation 404, scenes within the game play session being played by the player are detected. As mentioned above, the assist AI engine 108 accesses AI model 110 to identify scenes appearing in the game play. Scenes appearing in the game play are identified because AI model 110 is trained using various types of training material from previous game play sessions. Previous game material can be saved as game play sessions, state data, records, social media website video clips, screenshots, online game history, and aggregations thereof.

[0055] The more a game is played and the more access to and processing of game-related content occurs, the more diverse and numerous game scenarios that can appear in the game will become the AI ​​model 110. It should be understood that the learned game scenarios are game-specific. Each game will have different game scenarios, and an AI model 110 can be instantiated and generated for each game, making each AI model 110 unique and specific to the particular game it is learning from. Thus, when a user plays the game and a scene appears during gameplay, the game state information generated during gameplay can be used by AI model 110 and auxiliary AI engine 108 to identify similar previously encountered scenes, in order to generate control input for the ghost character 113. The ghost character 113 can provide ghost assistance because it has learned, using AI model 110 and auxiliary AI engine 108, what types of movement, actions, interactions, or skills are needed to complete a specific game scene, and then displays the specific game scene for player 107 to view according to mode setting 116. Player 107 can then try to perform these interactive skills using player character 109.

[0056] In operation 406, the output of the auxiliary AI engine is used to apply control input to the ghost character. The control input provides interactive actions to the ghost character to make progress in the scene detected during gameplay. The control input is customized for the ghost character's mode settings. In one implementation, in operation 408, it is determined whether the mode settings should be switched automatically. The mode settings may switch automatically based on user profile settings, user-learned preferences, or recommendations made by an AI engine or logic that understands the levels and interactions that occur in the gameplay context.

[0057] If the mode switching is done automatically, the mode switching settings are applied, allowing the auxiliary AI engine to generate control inputs for the ghost character based on the mode settings. If the mode setting is automatic, the method determines whether manual mode settings 410 have been received. If manual mode settings have been received, mode switching settings 12 are applied, and the auxiliary AI engine can therefore generate output control inputs consistent with the mode settings. If it is determined that no mode settings have been received, the auxiliary AI engine will continue to output control inputs to guide the control of the ghost character during gameplay interactions.

[0058] Figure 4B Another example of a mode setting switch 420 providing control over the operation of the AI ​​engine 108 is shown. As illustrated, player input 422 can be used to provide mode setting switching. Player input can rely on: user interface selection screens, button presses, controller movements, verbal input, gestures, or any other input that can be received by the interface to provide the computer with an indication that a selection has been made. Mode settings can also be generated via automated input 424. Automated input 424 can be generated in response to settings in the player's user profile. In other embodiments, the user profile can be updated from time to time based on learned player metrics, player interactions, preferences, etc. This information can then be used to generate automated input 424. In yet another embodiment, machine learning can be used to determine when automated input 424 should be generated to change the mode settings.

[0059] Once the mode setting switch 420 is activated, the assistance AI engine 108 changes its method of providing assistance. The method of providing assistance can take the form of global assistance rules, such as following a guided mode or a full mode. In other implementations, more style modes can be selected to determine and identify the style of providing assistance. The assistance AI engine 108 will receive input from an AI model 110, which continuously learns from training material associated with the game being played. In some implementations, the AI ​​model 110 is player-specific, making the assistance more tailored to the player's style. In another implementation, the AI ​​model 110 is more global, allowing access to more video and media from any number of sources to identify game scenes within the game the player is playing.

[0060] In this example, the game-assisted AI engine 108 can operate in multiple mode settings. Example mode settings (not limited to other types envisioned in this disclosure) include story mode 430, battle mode 432, exploration mode 434, and full game mode 436. Based on the mode setting selected by the player or automatically, the control inputs will vary, such as control inputs A, B, C, and D. These control inputs will differ because the ghost character 113 will be controlled via control inputs as shown by the dotted stick figure.

[0061] The dotted stick figure will provide illustrations for corresponding interactive actions A, B, C, and D based on the mode settings. As mentioned above, the interactive actions performed by the ghost character 113 will be consistent with the control input provided by the auxiliary AI engine 108, which is then implemented in the game as a ghost overlay character. In another embodiment, the ghost character 113 does not have to be a dotted line or a ghost figure. For example, the ghost character 113 can be a custom character or role from another game, or a fictional character from a movie.

[0062] For example, the ghost character could be an animated representation of the omniscient Master Yoda from Star Wars. Master Yoda could not only provide interactive movement to demonstrate specific scenes of how to play the game, but also provide verbal communication to player character 109, which could be heard by player 107. Therefore, ghost assistance could be a combination of movement and interactive actions performed by the ghost character, or it could take the form of provided speech or gestures visible to player 107 who controls player character 109 in the game scene during a game session. If the speech and gestures are provided by the animated character Master Yoda, then the visual representation of Master Yoda could be rendered as Master Yoda responding to the player-controlled character.

[0063] In this way, the verbal feedback provided by the ghost character 113 (whether in the form of an animated character, a ghost object, or a character) will assist the player in completing tasks, levels, interactive movements, or generally build skills for playing the game during gameplay.

[0064] In another implementation, players can also ask questions of the ghost character. Players can provide verbal input via the audio channel to request help completing specific scenes. For example, a player can say, " How do I jump across this river? "Then, the assistive AI engine will rely on the AI ​​model to determine how to cross the river, and the ghost character can respond to the player to explain how to cross it. In another implementation, the ghost character can simply demonstrate how to cross the river. This provides an interactive method for the player character to talk to the ghost character and use it as an interactive assistive robot while playing the game. It should be understood that the ghost assistance can be turned on or off according to the player's instructions. Therefore, the player does not have to continue playing in ghost mode, but can pause ghost mode for certain interactive scenarios and then resume it in ghost mode."

[0065] Furthermore, other implementations for providing ghost assistance may include providing color indicators. These color indicators can be ghostly, shadowy, or overlaid, assisting the player in identifying content relevant to progress in the game. For example, color indicators may be provided by ghost assistance logic to highlight doors providing exits to rooms, objects the player needs to capture, game controller inputs to select, or combinations thereof. These color indicators may also be integrated into the ghost character 113. The ghost character 113 can change color based on the urgency of the assistance provided. For example, if the assistance is very urgent, the ghost character may turn red. If the assistance is not urgent, the ghost character may simply be green or a shade thereof. In yet another implementation, assistance may also take the form of text and commentary floating above the game scene. The text may provide a description of what needs to happen to make progress or what movements the player needs to make. In yet another implementation, the provision of text bubbles, icons, chat, voice, and interactive movement icons may be mixed together to provide denser and more comprehensive assistance.

[0066] As mentioned above, the assistance provided through Ghost Mode can be integrated into the specific game logic of the game. In another implementation, it can be executed through the system logic of a system used for playing games. In other implementations, the assistance logic can be integrated into middleware that can be used for traditional games, including new games and the current game.

[0067] Figure 5Components of an example apparatus 500 that can be used to perform various embodiments of the present disclosure are shown. This block diagram illustrates apparatus 500, which may be combined with or may be a personal computer, video game console, personal digital assistant, server, or other digital device suitable for practicing embodiments of the present disclosure. Apparatus 500 includes a central processing unit (CPU) 502 for running software applications and optionally an operating system. CPU 502 may consist of one or more homogeneous or heterogeneous processing cores. For example, CPU 502 is one or more general-purpose microprocessors having one or more processing cores. Further embodiments may be implemented using one or more CPUs having a microprocessor architecture particularly suitable for highly parallel and computationally intensive applications, such as processing operations that interpret queries, identify context-dependent resources, and immediately implement and render context-dependent resources in video games. Apparatus 500 may be local to the player playing a game segment (e.g., a game console), remote to the player (e.g., a back-end server processor), or one of many servers in a game cloud system that uses virtualization to remotely stream game entertainment to clients.

[0068] Memory 504 stores applications and data used by CPU 502. Storage 506 provides non-volatile storage and other computer-readable media for applications and data, and may include fixed disk drives, removable disk drives, flash memory devices, and CD-ROMs, DVD-ROMs, Blu-ray, HD-DVDs, or other optical storage devices, as well as signal transmission and storage media. User input device 508 transmits user input from one or more users to device 500. Examples of such devices may include a keyboard, mouse, joystick, touchpad, touchscreen, still or video recorder / camera, gesture tracking device, and / or microphone. Network interface 514 allows device 500 to communicate with other computer systems via electronic communication networks and may include wired or wireless communication via local area networks and wide area networks (such as the Internet). Audio processor 512 is adapted to generate analog or digital audio output using instructions and / or data provided by CPU 502, memory 504, and / or storage device 506. The components of device 500 (including CPU 502, memory 504, data storage device 506, user input device 508, network interface 510, and audio processor 512) are connected via one or more data buses 522.

[0069] The graphics subsystem 520 is further connected to the data bus 522 and components of the device 500. The graphics subsystem 520 includes a graphics processing unit (GPU) 516 and a graphics memory 518. The graphics memory 518 includes display memory (e.g., a frame buffer) for storing pixel data for each pixel of an output image. The graphics memory 518 may be integrated into the same device as the GPU 508, connected to the GPU 516 as a separate device, and / or implemented within memory 504. Pixel data may be provided directly from the CPU 502 to the graphics memory 518. Alternatively, the CPU 502 may provide the GPU 516 with data and / or instructions defining a desired output image, and the GPU 516 may generate pixel data for one or more output images based on said data and / or instructions. The data and / or instructions defining the desired output image may be stored in memory 504 and / or graphics memory 518. In one embodiment, the GPU 516 includes 3D rendering capabilities that generate pixel data for an output image from instructions and data defining the geometry, lighting, shading, texture, motion, and / or camera parameters of a scene. GPU 516 may also include one or more programmable execution units capable of executing shader programs.

[0070] The graphics subsystem 514 periodically outputs pixel data for an image from the graphics memory 518 for display on the display device 510. The display device 510 can be any device capable of displaying visual information in response to signals from the device 500, including CRT, LCD, plasma, and OLED displays. The device 500 can provide, for example, analog or digital signals to the display device 510.

[0071] It should be noted that access services delivered over vast geographical areas (such as providing access to games in current implementations) often utilize cloud computing. Cloud computing is a computing paradigm in which dynamically scalable and often virtualized resources are provided as a service via the internet. Users do not need to be experts in the technical infrastructure supporting their “cloud.” Cloud computing can be divided into different services, such as Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). Cloud computing services typically provide commonly used applications (such as video games) online, accessible from a web browser, while the software and data are stored on servers in the cloud. Based on how the internet is depicted in computer network diagrams, the term cloud is used as a metaphor for the internet and is an abstraction of its hidden, complex infrastructure.

[0072] In some implementations, a game server can be used to operate a platform for recording video game player duration information. Most video games played over the internet operate via a connection to a game server. Typically, the game uses a dedicated server application that collects data from players and distributes it to other players. In other implementations, the video game can be executed by a distributed game engine. In these implementations, the distributed game engine can run on multiple processing entities (PEs), such that each PE executes a functional segment of a given game engine on which the video game runs. The game engine simply treats each processing entity as a computing node. The game engine typically performs a diverse range of operations to execute the video game application and additional services for the user experience. For example, the game engine implements game logic, performs game calculations, physics effects, geometric transformations, rendering, lighting, shading, audio, and additional in-game or game-related services. Additional services may include, for example, messaging, social utilities, audio communication, game replay functionality, help functions, etc. While game engines can sometimes run on an operating system virtualized by a hypervisor of a specific server, in other implementations, the game engine itself is distributed across multiple processing entities, each of which may reside on a different server unit in a data center.

[0073] According to this implementation, depending on the needs of each game engine segment, the corresponding processing entity used to perform the operations can be a server unit, a virtual machine, or a container. For example, if a game engine segment is responsible for camera transformations, a virtual machine associated with a graphics processing unit (GPU) can be provided to this particular game engine segment, as it will perform a large number of relatively simple mathematical operations (e.g., matrix transformations). Processing entities associated with one or more higher-power central processing units (CPUs) can be provided to other game engine segments that require fewer but more complex operations.

[0074] By using a distributed game engine, the game engine is configured with elastic computing properties that are not constrained by the capabilities of physical server units. Instead, more or fewer compute nodes are provided to the game engine as needed to meet the demands of the video game. From the perspective of video games and video game players, a game engine distributed across multiple compute nodes is no different from a non-distributed game engine that runs on a single processing entity, because the game engine manager or supervisor distributes the workload and seamlessly integrates the results to provide the video game output components to the end user.

[0075] Users access remote services using client devices, which include at least a CPU, display, and I / O. Client devices can be PCs, mobile phones, laptops, PDAs, etc. In one embodiment, a network operation on the game server identifies the type of device used by the client and adjusts the communication method accordingly. In other cases, the client device uses standard communication methods (such as HTML) to access applications on the game server via the Internet. It should be understood that a given video game or game application can be developed for a specific platform and a specific associated controller device. However, when such games are made available via a game cloud system as presented herein, users may access the video game with different controller devices. For example, a game may have been developed for a game console and its associated controllers, while the user may access a cloud-based version of the game from a personal computer using a keyboard and mouse. In such scenarios, input parameter configuration can define a mapping from inputs generated by the user's available controller device (in this case, a keyboard and mouse) to inputs acceptable for the execution of the video game.

[0076] In another example, users can access the cloud gaming system via a tablet computing device, a touchscreen smartphone, or other touchscreen-driven device. In this case, the client device and controller device are integrated into the same device, where input is provided using detected touchscreen inputs / gestures. For such devices, input parameter configuration can define specific touchscreen inputs corresponding to game inputs for the video game. For example, during the operation of the video game, buttons, directional pads, or other types of input elements can be displayed or overlaid to indicate the locations on the touchscreen that the user can touch to generate game inputs. Gestures (such as swipes in a specific direction or specific touch movements) can also be detected as game inputs. In one implementation, the user can be provided with instructions on how to provide game-specific inputs via the touchscreen, for example, before starting gameplay of the video game, so as to familiarize the user with the operation of controls on the touchscreen.

[0077] In some implementations, the client device acts as a connection point for the controller device. That is, the controller device communicates with the client device via a wireless or wired connection to send input from the controller device to the client device. The client device can then process this input and send the input data to the cloud gaming server via a network (e.g., via a local networked device such as a router). However, in other implementations, the controller itself can be a networked device with the ability to transmit input directly to the cloud gaming server via the network, without first transmitting such input through the client device. For example, the controller can connect to a local networked device (such as the router mentioned above) to send and receive data from the cloud gaming server. Therefore, while the client device may still be required to receive video output from the cloud-based video game and render it on a local display, input latency can be reduced by allowing the controller to send input directly to the game cloud server via the network, thus bypassing the client device.

[0078] In one implementation, the networked controller and client device can be configured to send certain types of input directly from the controller to the cloud gaming server, and other types of input via the client device. For example, input whose detection does not rely on any additional hardware or processing outside the controller itself can be sent directly from the controller to the cloud gaming server via the network, bypassing the client device. Such inputs may include button inputs, joystick inputs, embedded motion detection inputs (e.g., accelerometers, magnetometers, gyroscopes), etc. However, inputs utilizing additional hardware or requiring processing by the client device can be sent to the cloud gaming server by the client device. These may include video or audio captured from the game environment, which can be processed by the client device before being sent to the cloud gaming server. Additionally, input from the controller's motion detection hardware can be processed by the client device in conjunction with captured video to detect the controller's position and movement, which the client device then transmits to the cloud gaming server. It should be understood that the controller device according to various embodiments may also receive data (e.g., feedback data) from the client device or directly from the cloud gaming server.

[0079] In one implementation, various technical examples can be achieved using a virtual environment via a head-mounted display (HMD). An HMD can also be referred to as a virtual reality (VR) head-mounted device. As used herein, the term "virtual reality" (VR) generally refers to the interaction of a user with a virtual space / environment, involving viewing the virtual space through an HMD (or VR head-mounted device) in a way that responds in real-time to the movement of the HMD (controlled by the user) to provide the user with a sense of being in a virtual space or metaverse. For example, a user can see a three-dimensional (3D) view of the virtual space when facing a given direction, and when the user turns to one side and thus similarly rotates the HMD, the view of that side in the virtual space is then rendered on the HMD. An HMD can be worn in a manner similar to glasses, goggles, or a helmet and is configured to display video games or other metaverse content to the user. An HMD can provide a highly immersive experience to the user by means of its display mechanism being positioned close to the user's eyes. Therefore, an HMD can provide a display area occupying most or even the entire field of vision for each of the user's eyes and can also provide a viewing experience with three-dimensional depth and perspective.

[0080] In one implementation, the HMD may include a gaze-tracking camera configured to capture images of the user's eyes as the user interacts with the VR scene. The gaze information captured by the gaze-tracking camera may include information related to the user's gaze direction and specific virtual objects and content items in the VR scene that the user is interested in or interested in interacting with. Therefore, based on the user's gaze direction, the system can detect specific virtual objects and content items that the user may be interested in (where the user is interested in interacting with and participating in), such as game characters, game objects, game props, etc.

[0081] In some implementations, the HMD may include an externally facing camera configured to capture images of the user's real-world space, such as the user's body movements and any real-world objects that may be located in that space. In some implementations, the images captured by the external camera may be analyzed to determine the position / orientation of real-world objects relative to the HMD. Using the known position / orientation of the HMD, real-world objects, and inertial sensor data from them, the user's postures and movements during user interaction with the VR scene can be continuously monitored and tracked. For example, when interacting with a scene in a game, a user may make various postures, such as pointing and walking towards specific content items in the scene. In one implementation, the system may track and process postures to generate predictions of interaction with specific content items in the game scene. In some implementations, machine learning may be used to facilitate or assist the predictions.

[0082] During HMD usage, various one-handed and two-handed controllers can be used. In some implementations, the controller itself can be tracked by tracking lights included in the controller or by tracking shape, sensor, and inertial data associated with the controller. Using these different types of controllers, or even simply by using gestures made and captured by one or more cameras, it is possible to dock, control, manipulate, interact with, and participate in a virtual reality environment or metaverse rendered on the HMD. In some cases, the HMD can be wirelessly connected to a cloud computing and gaming system via a network. In one implementation, the cloud computing and gaming system maintains and executes a video game played by the user. In some implementations, the cloud computing and gaming system is configured to receive input from the HMD and interface objects via a network. The cloud computing and gaming system is configured to process the input to influence the game state of the executing video game. Output from the executing video game (such as video data, audio data, and haptic feedback data) is sent to the HMD and interface objects. In other implementations, the HMD can wirelessly communicate with the cloud computing and gaming system via alternative mechanisms or channels such as cellular networks.

[0083] Furthermore, while implementations in this disclosure may be described with reference to head-mounted displays, it should be understood that in other implementations, non-head-mounted displays may be used, including but not limited to portable device screens (e.g., tablets, smartphones, laptops, etc.) or any other type of display that can be configured to render video and / or provide displays of interactive scenes or virtual environments according to this implementation. It should be understood that the various embodiments defined herein can be combined or assembled into specific implementations using the various features disclosed herein. Therefore, the examples provided are merely some possible examples and are not limited to a variety of implementations that can be defined by combining various elements. In some examples, some implementations may include fewer elements without departing from the spirit of the disclosed or equivalent implementations.

[0084] The embodiments of this disclosure can be practiced with various computer system configurations, including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. The embodiments of this disclosure can also be practiced in distributed computing environments, where tasks are performed by remote processing devices linked via wired or wireless networks.

[0085] Although the method operations are described in a specific order, it should be understood that other housekeeping operations may be performed between operations, or operations may be adjusted so that they occur at slightly different times, or operations may be distributed throughout the system. As long as the processing of telemetry and game state data for generating the modified game state is performed in the desired manner, the system allows processing operations to occur at various intervals associated with the processing.

[0086] One or more embodiments can also be made into computer-readable code on a computer-readable medium. A computer-readable medium is any data storage device capable of storing data that can subsequently be read by a computer system. Examples of computer-readable media include hard disk drives, network attached storage (NAS), read-only memory, random access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tape, and other optical and non-optical data storage devices. Computer-readable media may include computer-readable tangible media distributed across network-coupled computer systems, enabling the storage and execution of computer-readable code in a distributed manner.

[0087] In one implementation, the video game is executed locally on a game console, a personal computer, or on a server. In some cases, the video game is executed by one or more servers in a data center. When a video game is executed, some instances of the video game can be simulations of the video game. For example, the video game can be executed by an environment or server that generates a simulation of the video game. In some implementations, the simulation is an instance of the video game. In other implementations, the simulation can be generated by an emulator. In either case, if the video game is represented as a simulation, the simulation can be executed to render interactive content that can be interactively streamed, executed, and / or controlled by user input.

[0088] Although the foregoing embodiments have been described in slightly more detail for clarity of understanding, it will be apparent that certain variations and modifications may be practiced within the scope of the appended claims. Therefore, the embodiments of the invention are to be considered illustrative rather than restrictive, and are not limited to the details given herein, but may be modified within the scope of the appended claims and their equivalents.

Claims

1. A method for providing assistance to a player during gameplay of a video game, the method comprising: A video game is executed for a game session, during which the player controls a player character. The selection of ghost assistants is based on the aforementioned game session detection; The game session is linked to an auxiliary artificial intelligence (AI) engine, which operates to provide ghost assistance to the player according to mode settings, wherein the ghost assistance is provided via a ghost character; During the game session, the ghost character is rendered to perform interactive actions in the game, the interactive actions being generated in response to control input generated by the auxiliary AI engine, and the interactive actions of the ghost character being configured to cause the ghost character to progress along the interactive path of the game; The ghost character assists the player interacting with the player character along the interactive path, and the ghost character's interactive actions are visible to the player, thus providing the ghost assistance.

2. The method of claim 1, wherein the AI ​​engine uses an AI model trained using game play assets associated with the game.

3. The method according to claim 2, wherein the game entertainment materials are processed to extract contextual features, which are then used by the AI ​​model for classification so that the AI ​​model can recognize game scenes.

4. The method of claim 3, wherein the control input generated by the auxiliary AI engine is used to control the ghost character to perform the interactive action in response to one or more game scenes in the game session, in order to make progress in the game.

5. The method of claim 1, wherein the AI ​​engine uses an AI model trained with game play material associated with previous play of the game to identify game scenes, the game scenes being used by the auxiliary AI engine to generate control inputs for enabling the ghost character to make progress in the game scenes appearing in the game session being played by the player.

6. The method of claim 5, wherein the mode setting defines the type of assistance provided by the ghost character during the ghost assistance.

7. The method of claim 6, wherein the mode setting is configured to switch during the game session.

8. The method of claim 7, wherein switching to another mode setting changes the type of assistance provided by the ghost character, and wherein the mode setting is selected from one of story mode, battle mode, exploration mode, and full game mode.

9. The method of claim 1, wherein the AI ​​engine uses an AI model trained with game play assets associated with previous play of the game to identify game scenes, the game scenes being used by the auxiliary AI engine to generate control inputs for enabling the ghost character to make progress in game scenes appearing in the game session being played by the player, and wherein the control inputs change upon detecting a mode setting switch.

10. The method of claim 9, wherein the mode setting switch focuses the ghost character on different play scenarios in which the ghost assists the player or the selected game narrative.

11. The method of claim 9, wherein the ghost assistance is a guided mode in which the player character is guided along the interactive path.

12. The method of claim 9, wherein the ghost assistance is a full mode in which the ghost character completes one or more interactive tasks for the player character along the interactive path.

13. The method of claim 9, wherein the game assets are selected from one or more of the following: user-generated content (UGC) game entertainment assets; assets containing game narrative content; assets containing game commission content; assets containing achievement content; assets containing character type content; assets containing storyline content; assets containing in-game skins; assets containing game objects; game assets captured within or outside the network; assets containing user setting data; assets containing privacy control data; assets containing age settings; or a combination of two or more of the above.

14. A computer-readable medium for providing assistance to a player during gameplay of a video game, the computer-readable medium comprising: Program instructions for executing video game commands for a game session, during which the player controls a player character; Program instructions for detecting the selection of ghost assist in the game session; Program instructions for linking the game session with an auxiliary artificial intelligence (AI) engine, which operates to provide ghost assistance to the player according to mode settings, wherein the ghost assistance is provided via a ghost character; Program instructions for rendering interactive actions performed by the ghost character in the game during the game session, the interactive actions being generated in response to control input generated by the auxiliary AI engine, the interactive actions of the ghost character being configured to cause the ghost character to progress along an interactive path in the game; The ghost character assists the player interacting with the player character along the interactive path, and the ghost character's interactive actions are visible to the player, thus providing the ghost assistance.

15. The computer-readable medium of claim 14, wherein the AI ​​engine uses an AI model trained using game play assets associated with the game.

16. The computer-readable medium of claim 15, wherein the game play material is processed to extract contextual features for classification by the AI ​​model so that the AI ​​model can recognize game scenes.

17. The computer-readable medium of claim 16, wherein the control input generated by the auxiliary AI engine is used to control the ghost character to perform the interactive action in response to one or more game scenes in the game session to achieve progress in the game.

18. The computer-readable medium of claim 14, wherein the AI ​​engine uses an AI model trained with game play material associated with previous play of the game to identify game scenes, the game scenes being used by the auxiliary AI engine to generate control inputs for enabling the ghost character to make progress in the game scenes appearing during the game session being played by the player.

19. The computer-readable medium of claim 18, wherein the mode setting defines the type of assistance provided by the ghost character during the ghost assistance.

20. The computer-readable medium of claim 19, wherein the mode setting is configured to switch during the game session, wherein switching to another mode setting causes a change in the type of assistance provided by the ghost character, and wherein the mode setting is selected from one of story mode, battle mode, exploration mode, and full game mode.

Images