Importing agent personalization data to instantiate personalization agent in user game session

Abstract

Aspects of the present disclosure relate to a personalized proxy service that generates and evolves a customized proxy that can be instantiated in a game to play with a user. A machine learning model is trained to control interaction of an agent with a game environment and a user during a game manipulation. A user may request a personalized agent to join the user's game manipulation session. A user device sends a request for a personalized agent to a gaming platform. The gaming platform determines whether the user has permission to execute a second instance of the game. When the user has permission to execute a second instance of the game, the second instance of the game may be executed on the user device. Information received from the personalized proxy service is used to instantiate a personalized proxy in a second instance of the game.

Description

Background Art

[0001] The gaming industry makes up a large portion of the technology sector. As technology moves towards an increasingly connected future, gaming will be a core component of that connected future. Artificial intelligence has traditionally been a major component of the gaming industry, however, as AI development continues to advance, it has become more difficult for the gaming industry to incorporate that advancement into gaming. For example, it is difficult for game development studios to incubate AI because most advanced AI systems require a lot of custom domain experience.

[0002] The various aspects disclosed herein are made with respect to these and other general considerations.In addition, although relatively specific problems may be discussed, it should be understood that the examples should not be limited to solving specific problems identified in the background or elsewhere in this disclosure. Summary of the invention

[0003] Various aspects of the present disclosure relate to a personalized agent service that generates and evolves customized agents that can be instantiated in a game to play with a user. Machine learning models are trained to control the agent's interaction with the game environment and the user during game play. As the user continues to play with the agent, one or more machine learning models develop a game play style for the agent, which complements the user's preferred play style, incorporates the user's preferred strategies, and is generally customized for interaction with the user. The agent personalization data generated during game play is stored by the service, allowing the user to import the agent in different games, thereby creating an unchanging game play partner that the user can play with in a variety of different games without requiring the different games to include technology that can develop AI controlled agents.

[0004] In another aspect, a user device that allows a user to request a personalized agent to join the user's game manipulation session is disclosed. The user device sends a request for the personalized agent to a game platform. The game platform determines whether the user has permission to execute a second instance of the game. When the user has permission to execute the second instance of the game, the second instance of the game can be executed on the user device. Information received from the personalized agent service is used to instantiate the personalized agent in the second instance of the game. A communication session is established between the device executing the second instance of the game and the personalized agent service. The personalized agent service uses one or more machine learning models to analyze game data to determine a current game state. Based on the current game state, one or more actions are determined to control the game manipulation of the personalized agent.

[0005] This summary is provided to introduce a selection of concepts in simplified form, which will be further described in the following detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additional aspects, features, and / or advantages of the examples will be set forth in part in the following description, and in part will be apparent from the description, or may be learned by practice of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Non-limiting and non-exhaustive examples are described with reference to the following figures.

[0007] Figure 1A An overview of an example system for generating and using a user-personalized agent in a gaming system is presented.

[0008] Figure 1B An overview of an example system for generating and utilizing personalized agent responses in a gaming system is shown.

[0009] Figure 2 An example of a method for generating a personalized agent is shown.

[0010] Figure 3 An exemplary system for instantiating a personalized agent to engage in gaming manipulation with a user is depicted.

[0011] Figure 4 Exemplary methods for creating a personalized agent game session and instantiating a personalized agent in the agent game session are described.

[0012] Figure 5 An exemplary method for determining whether a personalized agent gaming session can be established is depicted.

[0013] Fig. 6A An exemplary method 600 for instantiating and controlling a personalized agent within a personalized agent game session is depicted.

[0014] Figure 6B An example of a method utilizing computer vision to enable a personalized agent to interact with a user during game play is depicted.

[0015] Figure 7 is a block diagram illustrating example physical components of a computing device with which aspects of the present disclosure may be practiced.

[0016] Fig. 8A is a simplified diagram of a mobile computing device in which aspects of the present disclosure may be implemented.

[0017] Figure 8B is another simplified block diagram of a mobile computing device with which aspects of the present disclosure may be practiced. DETAILED DESCRIPTION

[0018] Various aspects of the present disclosure are described more fully below with reference to the accompanying drawings, which form a part thereof and illustrate specific example aspects. However, different aspects of the present disclosure can be implemented in many different ways and should not be construed as limited to the aspects set forth herein; on the contrary, these aspects are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of various aspects to those skilled in the art. Practical aspects can be methods, systems, or devices. Therefore, various aspects can take the form of hardware implementations, complete software implementations, or implementations combining software and hardware aspects. Therefore, the following detailed description should not be taken in a limiting sense.

[0019] Various aspects of the present disclosure provide systems and methods for using machine learning techniques to provide personalized agents or robots that can be used in games or other types of environments. In other examples, various aspects disclosed herein relate to: using reinforcement learning agents that can be trained via computer vision without relying on deep game hooks; using models (e.g., base models, language models, computer vision models, speech models, video models, audio models, multimodal machine learning models, etc.) to train agents so that the agents are operable to understand written and spoken text, have a defined knowledge system, and can write their own code; providing agents that interpret state information using computer vision and audio prompts, providing agents that receive user instructions based on computer vision and audio prompts, and providing systems in which user feedback is used to improve future interactions with agents across different games and applications.

[0020] In an example, a generative multimodal machine learning model processes user input and generates a multimodal output. For example, a dialog agent according to the aspects described herein may receive user input such that the user input may be processed using a generative multimodal machine learning model to generate a multimodal output. The multimodal output may include natural language output and / or programmable output, among other examples. The multimodal output may be processed and used to affect the state of an associated application. For example, at least a portion of the multimodal output may be executed or may be used to call an application programming interface (API) of an application. In some examples, a generative multimodal machine learning model (also generally referred to herein as a multimodal machine learning model) used according to the aspects described herein may be a generative transformer model. In some cases, explicit and / or implicit feedback may be processed to improve the performance of the multimodal machine learning model.

[0021] In an example, user input and / or model output is multimodal, as used herein, which may include one or more types of content. Example content includes, but is not limited to, spoken or written language (which may also be referred to herein as "natural language output"), code (which may also be referred to herein as "programmed output"), images, video, audio, gestures, visual features, intonation, contour features, posture, style, font, and / or transitions, among other examples. Thus, in contrast to machine learning models that process natural language input and generate natural language output, aspects of the present disclosure may process input and generate output having any of a variety of content types.

[0022] In doing so, the systems and methods disclosed herein enable the personalized agent to learn who the user is (also interchangeably referred to herein as a "player"), how the user speaks, what the user's strategy is, and how the user plays. It retains a "memory" of past interactions with the user and acts as a constant companion to the user as the user participates in different experiences (such as playing different games, playing different game modes, etc.). To do so, the various aspects disclosed herein are operable to store meta-classifications (e.g., sentiment analysis, intent analysis, etc.) associated with the user's personal data, provided the user has been given permission to do so.

[0023] Fig. 1 shows an overview of an example system 100 for generating and using a user personalized agent in a game system. As depicted in Fig. 1, a user device 102 interacts with a cloud service 104, which hosts a game service 106 (or other types of applications) and an instantiation of an agent 108 that can interact with a game. The game device can be a console game system, a mobile device, a smart phone, a personal computer, or any other type of device that can execute a game locally or access a hosted game on a server. In one example, a game associated with a game service 106 can be directly hosted by a cloud service 104. In an alternative example, a user device can host and execute a game locally, in which case the game service 106 can serve as an interface that promotes the communication between one or more instantiated agents 108 and the game. A personalized agent library 107 can store and execute components for one or more agents associated with a user of the user device 102. The components of the personalized agent library 107 can be used to control the instantiated agent 108.

[0024] In the example, based on the text communication, voice command and / or player action received from the user device 102, one or more agents from the personalized agent library 107 interact with the game via instantiation agent 108. That is, the system 100 supports the interaction with the agent, as if they were other human players, or alternatively, as the player will usually interact with the traditional in-game NPC. In doing so, one or more agents hosted by the agent library 108 are operable to interact with the different games played by the user, without the need to change the game. In other words, the system 100 is operable to work with the game, and does not need to specifically develop the game to support the agent (for example, the agent does not need API access, and the game does not have to interact with the agent with specific code development, etc.). In doing so, the system 100 provides an extensible solution, which allows users to play with customized agents across various different games. In other words, using audio and visual interaction and / or disclosed API, the game state of the game is transmitted between one or more instantiation agents 108a via game services 106. In doing so, the instantiated agent 108 can interact with the game in the same way as the user (e.g., by interpreting the video, audio and / or tactile feedback from the game) and / or in the same way as the NPC in the game (e.g., via a public API). Similarly, one or more agents can interact with the user playing the game using the game device 102 as another player or in a manner similar to the NPC interacting with the user. That is, one or more agents can receive visual, text or voice input from the user playing the game via the game service 106, and / or receive game information (e.g., current game state, NPC list, NPC capabilities, etc.) via a public API. In this way, the system 100 allows users playing games via the user device 102 to interact with one or more agents, just as they would interact with any other player on the NPC, however, based on the past interactions with the user in the current game and other games, the instantiated agent 108 can be personalized for the user. In order to facilitate this type of user interaction, one or more instantiated agents 108 can use computer vision, speech recognition or other known technologies when processing interactions with the user, so as to interpret commands received from the user and / or generate response actions based on user actions.

[0025] For example, consider that a user plays a first-person shooter using a personalized agent. The user may say "cover the right side". The speech recognition model can be one of the models 118, which can be used to interpret the audio received from the user as a modality that can be understood by the agent. In response, the agent 108 can occupy a position on the right side of the user, or otherwise perform an action to cover the right side of the user in response to the user's audio instructions. As another example, consider an action role-playing game, in which two targets must be defended at the same time. The agent playing with the user can use computer vision to analyze the current view of the game to determine that there are two targets (e.g., identify two targets on the map, explain the displayed task log indicating that two targets must be defended, etc.). Similarly, computer vision can be used to identify the user's player character and determine that the player character is heading for the first target. Based on feedback from the computer vision model, the agent can instruct its representative character to move to the second target and defend the second target.

[0026] According to these examples, those skilled in the art will understand that, by adopting speech recognition, computer vision technology, etc., one or more agents from the agent library will be able to interact with the user in a manner similar to that of other users interacting during collaborative game manipulation. In doing so, one or more agents can be generated from the agent library independently of the specific game, because the instantiated agent 108 may not have to have API or programming access to the game to interact with the user. Alternatively, the instantiated agent 108 may have API access or programming access to the game, such as when the instantiated agent "owns" the NPC in the game. In the described environment, the agent can interact with the game state and the user based on access via API. Because the system 100 provides a solution that an agent can be implemented separately from an individual game, one or more agents in the agent library can be personalized to interact with a specific user. This personalization can be performed across different games. That is, over time, the agent learns details about the user, such as the user's likes and dislikes, the user's game style, the user's communication mode, the user's preferred strategy, etc., and can adapt to the user accordingly across different games.

[0027] Agent personalization can be generated and updated over time via the feedback collection engine 110. The feedback collection engine 110 receives feedback from the user and / or from the instantiated agent 108 executed in the game. The collected feedback may include information related to the following: the user's game style, user communication, user interaction with the game, user interaction with other players, user interaction with other agents, the results of the instantiated agent 108 actions performed in the game, the interaction between the player and the instantiated agent 108 actions in the game, or any type of information generated by the user device 102 when the user plays the game. In order to comply with user privacy considerations, information can only be collected by the feedback collection engine 110 when permission to do so is received from the user. The user can choose to join or exit the collection at any time. The collected data can be: implicit data, for example, data based on the user's normal interaction with the game; or explicit data, such as a specific command provided to the system by the user. An example of a specific command can be that the user instructs the agent to address the user by a specific character name. The data collected by the feedback collection engine 110 can be provided to the prompt generator 112.

[0028] The prompt generator 112 can use the data collected by the feedback collection engine 110 to generate prompts for personalizing one or more agents of the agent library 108. That is, the prompt generator 112 interprets the collected feedback data to generate instructions that can be executed by one or more agents to perform actions by the agent. The prompt generator is operable to generate new prompts or instructions based on the collected feedback, or to change existing prompts based on the newly collected feedback. For example, if the user initially plays the game as a frontline attacker, the prompt generator 112 can generate instructions that enable the instantiated agent 108 to implement a supported game style, such as becoming a remote attacker or a support character. If the user switches the game style to one of the remote damage initiators, the prompt generator 112 can identify the change via the feedback data and adjust the instantiated agent to adjust to the player's new style (e.g., switch to a frontline attacker to draw hatred or hostility from the player character). The instructions generated by the prompt generator are provided to the cloud service 104 to be stored by the cloud service 104 as part of the agent library 108, thereby storing meta-classifications associated with a specific user (e.g., sentiment analysis, intent analysis, etc.). In doing so, instructions generated based on the user's play style or preferences in the first game can be incorporated by the agent not only into the first game, but also into other games played by the user. That is, using the instructions generated by the prompt generator 112, the cloud service 104 can instantiate agents across a variety of different games that have been personalized for a particular user based on the user's previous interactions with the instantiated agent 108, regardless of whether the user is playing the same game or a different game. Although aspects described herein describe a separate prompt generator 112 as generating commands to control the instantiated agent 108, in alternative aspects, the commands can be generated directly by one or more machine learning models employed by the agent library 107, or generated via a combination of various different components disclosed herein.

[0029] However, in some scenarios, problems may be encountered during the training process of various machine learning models that can be used to generate personalized agents. For example, there may be a failed training session due to data or processing errors. In another example, the training process may fail due to user errors. For example, a user who is exposed to a new game may initially play the game incorrectly. The user's incorrect actions or experiences may train the personalized agent to play in a way that negatively affects the game manipulation. In this way, the system 100 may include a process for rolling back or resetting one or more machine learning models (or any other components disclosed herein) to correct errors that may occur when training one or more machine learning models or any other errors that may generally occur when developing personalized agents. For example, the system 100 can periodically maintain snapshots of different machine learning models (or other components) that save the state of the components at the time of the snapshot. This allows the system 100 to roll back all components, a subset of components, or a specific component in response to detecting a training error in the future. Multiple different snapshots can be storages representing the state when the personalized agent develops over time, thereby providing the system 100 (or user) with options to determine the appropriate state for rollback when an error is encountered.

[0030] The personalized agent library 107 may also include a fine-tuning engine 114 and one or more models 116. The fine-tuning engine 114 and the model 116 are operable to interact with user actions via the user device 102, the instantiated agent 108, and the game training model 124 to process feedback data received from various sources. Any number of different models may be employed, either individually or in combination, for use with the examples disclosed herein. For example, a base model, a language model, a computer vision model, a speech model, a video model, and / or an audio model may be employed by the system 100. As used herein, a base model is a model trained on a wide range of data that may be applicable to a wide variety of tasks (e.g., a model capable of handling a variety of different tasks or modalities).

[0031] One or more models 116 can process video, audio and / or text data received from a user or generated by the game during game play to interpret user commands and / or derive user intent based on user communications or in-game actions. Outputs from one or more models 116 are provided to a fine-tuning model 114, which can use the output to modify the prompts generated by the prompt generator 112 to further personalize the instructions generated for the user based on the agent's past interactions with the user. The personalized agent library 107 can also include a game data (legend) 122 component that is operable to store information about a variety of different games. The game data (legend) 122 component stores information about the game, game legends, story elements, available abilities and / or items, etc., which the agent can be used by other components (e.g., models 118, fine-tuning engine 114, prompt generator 112, etc.) to generate instructions to control the instantiated agent 108 according to the theme, story, requirements, etc. of the game.

[0032] The various components described herein can be utilized to interpret the current game state based on data received from the game (e.g., visual data, audio data, tactile feedback data, data disclosed by the game through an API, etc.). In addition, the components disclosed herein are operable to generate instructions based on the current game state to control personalized agent actions in the game. For example, the components disclosed herein are operable to generate instructions to control the interaction of the personalized agent with the game in a manner similar to how a human would interact with the game (e.g., by providing specific controller instructions, keyboard instructions, or any other type of interface that supports game manipulation control). Alternatively or additionally, the various components can generate code that controls how the personalized agent interacts in the game. For example, the code can be executed by the personalized agent, which causes the personalized agent to perform specific actions within the game.

[0033] The personalized agent library 107 may also include an agent memory component 120. The agent memory component may be used to store personalized data generated by various other components described herein as well as game styles, techniques, and interactions learned by the agent via past interactions with the user. The agent memory 120 may provide additional input to the prompt generator 112 for determining instantiated agent actions during game play.

[0034] So far, the components of the described system 100 have focused on the creation of personalized agents, the continuous evolution of personalized agents via continuous interaction with users, and the instantiation of personalized agents in user's game sessions. Although the personalization of agents for specific users provides many game manipulation benefits, instantiated agents also need and understand how to interact with games and play games. In one aspect, the components of the personalized agent library 107 are operable to learn and refine the game manipulation of agents based on sessions with users. However, when users play new games, the time required for agents to learn game manipulation mechanisms in order to become useful partners may not be feasible only by playing games with users. In order to solve this problem, the system 100 also includes a game manipulation training service 124, which includes a game library 126 and a game manipulation machine learning model 128. In various aspects, the game library includes any number of games supported by cloud services 104 and / or user devices 102. The game manipulation training service 124 is operable to perform sessions for various games stored in the game library 126, and instantiate agents in the executed games. The game manipulation machine learning model 128 is operable to receive data from the executed game and the agent actions performed in the game as input, and in response, generates control signals to guide the agent's game manipulation in the game. By using reinforcement learning, the game manipulation machine learning model is operable to develop an understanding of the game manipulation mechanism of both specific games and game genres. In doing so, the game manipulation training service 124 provides a mechanism in which the agent can be trained to play a specific game or a specific type of game without user interaction. The personalized agent library 107 is operable to receive a trained model (or interact with a trained model) from the game manipulation training service 124, which can be stored as a part of the agent memory, and adopts those models with other personalized components to control the instantiated agent 108. In doing so, the user's experience of interacting with the agent in the game is greatly enhanced, because the user does not need to invest time training the agent in the specific mechanism of the game. Additionally, by importing or interacting with a trained game play machine learning model 128 provided by the game play training service 124, the personalized agent library 107 is able to employ the trained instantiated agent 108 to play with the user when the user first launches a new game.

[0035] Those skilled in the art will appreciate that the system 100 provides a personalized agent or artificial intelligence that is operable to learn player identities, learn player communication styles or tendencies, learn strategies adopted and used by players in a variety of different games and scenarios, learn game manipulation mechanics for specific games and game genres, etc. In addition, one or more agents generated by the system 100 can be stored as part of a cloud service, which allows the system to retain a "memory" of a user's past interactions, thereby allowing the system to generate an agent that acts as a consistent user companion across different games without the need to specifically design the game to support such an agent.

[0036] Figure 1B An overview of an example system 150 for generating and utilizing personalized agent responses in a gaming system is shown. As shown in the system 150, two players—Player 1 152 and Player 2 154—interact with one or more agents 156. Although two players are shown, those skilled in the art will appreciate that any number of players may participate in a gaming session using the system 150. An assistant service 158 is provided that helps personalize the interaction of one or more agents 158 with an individual player or with multiple players simultaneously. Figure 1A As discussed in , one or more models 166 may be used to generate or modify prompts 168. The prompts 168 are provided to an assistant service 158, which applies multiple engines (e.g., an agent character engine 160, a user goal or intent engine 162, and a game lore or constraint engine 164) to modify the prompts to provide more personalized interactions with individual or group players.

[0037] For example, the agent character engine 160 can modify or adjust the prompt (or the action determined by the prompt) based on the personalized information associated with the agent. For example, the user can adopt an agent with a preferred personality. The agent character engine 160 can modify the prompt or the response generated by the prompt based on the agent's personality. The user intent or goal engine can modify the prompt (or the action determined by the prompt) based on the user's current goal or the intention behind the user's action or request. The user's goal or intention can change over time, can be based on a specified user goal, or can be determined based on the user's action. The game lore or constraint engine 164 can modify or adjust the prompt (or the action determined by the prompt) based on the characteristics of the game. For example, the agent can "own" a non-player character in the game (as discussed in further detail below). The game lore or constraint engine 164 can modify the prompt based on the NPC personality or restrictions. When modifying or adjusting the prompt, various engines of the assistant service can be used individually or in combination. The adjusted prompt is then provided to one or more associated agents 156 for execution.

[0038] Figure 2An example of a method 200 for generating a personalized agent is shown. For example, the method 200 can be adopted by the system 100. The process begins at operation 202, where a game session is instantiated, or an indication of a game session being instantiated is received. As described above, the game session can be hosted by a system executing the method 200, or can be hosted by a user device, such as on a game console. When the game is instantiated or an indication that a game session is established is received, the process continues to operation 204, where the agent is instantiated as part of the game session. In one example, the agent can be instantiated in response to receiving a request to add the agent to the game session. For example, a request can be received to instantiate an agent in a multiplayer game or an agent that controls an NPC or AI partner in a single-player game. Instantiating an agent can include identifying an agent from an agent library associated with a user playing the game. As described above, aspects of the present disclosure provide for generating agents that can play different games. Therefore, the agent instantiated at operation 204 can be instantiated using various components stored as part of a personalized agent library. For example, agents may be instantiated using personalized features learned over time through interaction with the game, game data or lore saved about a particular game or genre, machine learning models trained to perform mechanics and actions specific to a game in which the agent is instantiated or trained based on a genre of games similar to the game in which the agent will be instantiated. As previously discussed, the selected agent may be personalized for the user playing the game based on past interactions with the user in the same game or a different game than the game initiated at operation 202. Alternatively, rather than dynamically instantiating an agent using different components stored in an agent library, a specific agent may be selected at operation 204. That is, the agent library may contain specific "builds" for different types of agents that are designed by the user or derived through specific game manipulation with the user. These agents may be saved and instantiated by the user in future game sessions for the same game or a different game from the game in which they were originally created. When the agent is instantiated at operation 204, the agent joins the game session with the user.

[0039] At operation 206, the current game state is interpreted by audio and visual data and / or by API access granted to the agent by the game. As previously mentioned, certain aspects of the present disclosure provide the generation of agents that can interact with the game without API or programmatic access to the game. In this way, the instantiated agent interacts with the game in the same way as the user, that is, by audio and visual data associated with the game. Alternatively, for example, when the agent has an NPC, the agent can be granted API access to the game to interact with the game. At operation 206, various speech recognition, computer vision, object detection, OCR processes, etc. can be used to process communications received from players (e.g., verbal commands, text-based commands) and game states received through the currently displayed view (e.g., using computer vision) or via API to interpret the game state. Then, the current game state is used to generate agent actions at operation 208. For example, agent actions can be performed based on verbal commands received from the user. Alternatively, agent commands can be generated based on the current view. For example, if an enemy appears on the screen, computer vision and / or object detection may be used to identify the enemy, and a command for the agent to attack the enemy may be generated at operation 208. Although not shown, operations 206 and 208 may be performed continuously while the game session is active.

[0040] The process continues to operation 210, where user feedback is received. The received user feedback can be explicit. For example, a user can issue a specific command to the agent to perform an action or change the action they are currently performing. Alternatively or additionally, the user feedback can be implicit. Implicit user feedback can be feedback data generated based on user interaction with the game. For example, the user may not explicitly provide commands to the agent, instead, the user can adjust their actions or game style based on the current game state and / or in response to the actions performed by the agent. In an example, user feedback can be continuously collected during a game session. The collected feedback can be associated with a concurrent game state or agent action.

[0041] When user feedback is collected, the process continues to operation 212, where prompts are generated for one or more agents based on the user feedback. In an example, the generated prompt is an instruction to perform an agent action in response to the state of the game or a specific user interaction. Prompts can be generated using one or more machine learning models that receive user feedback and / or actions performed by one or more agents and / or existing prompts and / or state data. The output of the machine learning model can be used to generate one or more prompts. In an example, the machine learning model can be trained using information related to the user so that the output from the machine learning model is personalized for the user. Alternatively or additionally, the machine learning model can be trained for a specific game or application, for a specific user group (e.g., an e-sports team), etc. A plurality of machine learning models can be used to generate prompts at operation 212. In other examples, in addition to or in place of the use of a machine learning model at operation 212, other processes, such as rule-based processes, can be used. In addition, new prompts can be generated at operation 212, or existing prompts can be modified.

[0042] Once one or more prompts are generated at operation 212, flow proceeds to operation 214, where the one or more prompts are stored for future use by one or more agents. For example, the one or more prompts may be stored in an agent library. By storing the prompts generated at 212 with the agent library, the agent will be able to utilize the prompts to interact with the user across different games, thereby providing a personalized agent with which the user can play different games.

[0043] Figure 3 An exemplary system 300 for instantiating a personalized agent to engage in gaming manipulation with a user is depicted. For clarity of description, the system 300 is described as including specific devices that perform the described actions in a specific order. However, those skilled in the art will appreciate that other devices may be included as part of the system 300, or that the described actions may be performed in a different order, and / or that the described requests may originate from or be sent to different devices without departing from the scope of the present disclosure. Figure 3 As shown, system 300 may include user device 302, game platform 304, and personalized proxy service 306. Although not shown, in one aspect, user device 302, game platform 304, and personalized proxy service 306 may be separate devices that communicate via a network. Alternatively, components of user device 302, game platform 304, and personalized proxy service 306 may reside on the same device or device network. For example, game platform 304 and personalized proxy service 306 may be part of the same cloud network.

[0044] User device 302 may be a game console, a personal computer, a smart phone, a tablet computer, or any other type of device capable of executing a game application. Game platform 304 may be one or more servers or cloud networks that support game services. Exemplary services supported by game platform 304 may be, for example, hosting online multiplayer games, delivering digital media, managing licenses and authorizations, managing friend lists, allowing communication between players, etc. Examples of game platforms include, but are not limited to, XBOX LIVE, STEAM, PLAYSTATION NETWORK, BATTLE.NET, etc. Personalized agent service 306 may be a server or cloud network that can train and maintain a library of personalized agents that can be adopted by users in various different games. Although shown as separate entities or networks, in some aspects, personalized agent service 306 may be part of the same network as game platform 304 or share the same network with it.

[0045] The user device 302 can execute a player game instance 308 for the user. The player game instance can be any multi-player (or in some instances, single player) game. Users participating in the player game instance 308 can access a contact or friend list via the game instance, or a contact or friend list hosted by the game platform 304. For example, the user device 302 may include a game service interface / client 310 that allows the user to access their contact or friend list hosted by the game platform 304. In one example, the game service / client interface 308 can be a component of the user device 302 operating system, such as an operating system of a game console. Alternatively or additionally, the game service / client interface 308 can be a part of an application resident on the user device 308, such as a client-side application for the game platform 304. As an example, a user may wish to invite a friend to join a game session associated with the player game instance 308. In this way, a user can access their contact or friend list through the player game instance 308 or via the game service interface / client 310 to see if any of their friends are online. In some cases, a user may play a game at a time when none of their friends are around, none of them want to play the same game as the user, or none of the user's friends have the character or specific specifications required to play the game with the user. Through integration with the personalized agent service 306, aspects of the present disclosure may also display one or more personalized robots associated with the user as part of the user's contact or friend list. Thus, if the user cannot find a friend who can play or is willing to play, the user may invite one or more of their personalized agents to play with the user. In one example, the user may invite one or more personalized agents to join a game session, just as the user would invite other human players, for example, by selecting a specific agent from the user's friend list. For example, the user may have one or more predefined agents that can be selected directly from a friend list. Alternatively or additionally, in another example, as part of inviting a personalized agent, the user may select or provide additional parameters of the agent related to the agent's characteristics, such as specific character traits or abilities, specific strategies, specific game styles, specific roles, etc. These parameters may be associated with an invitation to allow the personalized agent to join the user's game session. The invitation to the agent may be provided to the game platform 304 via the game service interface / client 310.

[0046] As discussed herein, aspects of the present disclosure relate to allowing users to invite personalized agents to join users in game manipulation sessions without requiring individual games to explicitly support agent creation or agent game manipulation. Therefore, from the perspective of the game being played by the user, adding a personalized agent will be no different from adding a new human player to the game. Due to this, in order for a personalized agent to join the game, the personalized agent will need a new instance of the game to join, just as the human player who joins must access to play the game (e.g., by owning a copy of the game, having a license to the game, having a subscription to the game, etc.). The game platform 304 may include an agent invitation interface 312 operable to receive a request to invite an agent from a user device 302. In response to receiving a request to add a personalized agent from a user device 302, the game platform 304 may determine, via an agent invitation interface 312, for example, via the identification of the agent in the invitation, via additional parameters associated with the invitation, etc., to determine that the invitation is for a personalized agent (as opposed to another human player). When determining that the invitation is for a personalized agent, the game platform 304 may determine whether the user of the personalized agent has permission to play the game. For example, the license manager 314 can perform a search to determine whether the user has multiple licenses for the game associated with the request to invite the personalized agent. If the user has multiple licenses for the game, or has an additional license account for the game, the personalized agent can use one of the additional licenses to join the game. Alternatively, the game platform 304 or the game itself can provide or allow the user to purchase an agent license. The agent license can be limited to allowing agents (rather than other human players) to execute instances of the game in order to play games with the user. Since the license is limited to the agent, the agent license may be provided to the user at a cost different from the standard game license (for example, it may be lower than the standard game license, it may be higher than the standard game license, it may be a certain fee that will increase or decrease the initial game license as the user purchases the additional agent license, etc.). If the license manager 314 cannot find additional users or agent licenses, the game platform 304 can transmit a request to the user device 302, for example, via the agent invitation interface 312, so that the user device 302 prompts the user to purchase additional game licenses or agent licenses. When the additional game or user license is successfully purchased, the additional game or user license can be added to the user account, for example, via the license manager 304. When the user has an additional game license or an agent license, the license manager 314 can obtain any authorization and / or license information required to instantiate an additional game session for the agent, and the game platform 304 can send the authorization and / or license information to the user device 302, thereby allowing the user device 302 to instantiate a new instance of the game for the agent.

[0047] Once the appropriate permission is confirmed by the game platform or alternatively by the game itself, the game platform can generate a request for a personalized agent. For example, the game platform 304 can aggregate personalized agent information from the agent invitation request (e.g., a personalized agent identifier that identifies a specific personalized agent, a specific character feature or ability, a specific strategy, a specific game style, a specific role, etc.) from the request. This information can be included as part of a request to create and / or instantiate a personalized agent in a newly created game session. The game platform 304 can also access additional information about the game that the user or agent will join from the account / game data storage 316. The account / game data storage 316 can store information about the user, such as the user's ID, game nickname, and the user's player character information (e.g., category, role, statistics, characteristics, etc.). The user information can also be included in the request to create and / or instantiate a personalized agent. In another example, the game platform can also access specific information about the game, such as a game server or IP address to be associated with a game manipulation session, and account information associated with the character of the personalized agent in the game (e.g., personalized agent character level, ability, role, etc.). Information in the agent invitation request from the user device 302, user and / or game information from the account / game data store 316, and / or authorization information for the personalized agent 318 can be aggregated and packaged into a request to create and / or instantiate a personalized agent by the agent service interface 318. The agent service interface 318 can then send a request to create and / or instantiate a personalized agent to the personalized agent interface 306.

[0048] The personalized agent interface 306 is operable to receive a request to create and / or instantiate a personalized agent from the game platform 304 via the agent request interface 320. The agent request interface 320 is operable to extract request parameters (e.g., information related to the requested characteristics of the personalized agent, user information, and / or game information) and provide the request parameters to the personalized agent selector 322. The personalized agent selector 322 analyzes the request parameters and identifies the requested agent characteristics based on the request parameters. For example, the personalized selector can identify a request for a specific characteristic from the request parameters. In addition, the personalized selector 306 can infer characteristics based on the request parameters. For example, information about the game (such as the game type or game genre) can be used to identify agent characteristics related to the game or NPC. For example, if the game is a role-playing game, the personalized selector 306 can identify agent characteristics that are related to or useful for role-playing games. Similarly, if the game is a first-person shooter game, the personalized selector 306 can identify agent characteristics related to first-person shooter games. Similarly, the characteristics of the personalized agent character of the characters in the game can be used to infer relevant agent characteristics. For example, if the in-game character of the personalized agent is a healer, the personalization selector may identify agent characteristics that are relevant to the healer category. Upon identifying the relevant agent characteristics, agent data related to the relevant agent characteristics are retrieved from the personalized agent data store 324. In an example, the personalized agent data store 324 may store any type of personalized information previously generated for the agent (e.g., prompts, machine learning models, agent personality traits, etc.). The retrieved personalized information may be aggregated from the personalized agent data store 324 and disclosed to (e.g., sent to or provided to) the user device 302, for example, via the agent request interface 320.

[0049] After receiving the license and / or authorization information from the game platform 304, the user device 302 can execute a virtual machine on the client device. The virtual machine can be used to execute a second game instance for the agent character on the user device 302 while executing the player game instance 308. After establishing the virtual machine that executes the personalized agent game instance 328, the user device can create an instance 328 of the personalized agent based on the personalized agent data received within the game session executed within the virtual machine using the personalized agent information disclosed by the personalized agent service 306 via the agent request interface 320. In this way, the client device 302 may not execute two different instances of the game, one for the user's player character and one for the personalized agent's character. From the perspective of the game, there are now two different instances of the game operated by two different players. That is, the personalized agent can interact with the game without the game providing interactive support for the agent (for example, without creating a specific API to allow the personalized agent to interact with the game). In this way, the multi-player function of the game can be utilized without modification to allow the user to invite their personalized agent to the game session when the user's friends are not online or otherwise unable to play the game with the user. However, since the game does not need to disclose an API to allow agent interaction, the personalized agent can participate in the game and interact with it like a normal human player. That is, the personalized agent is operable to receive visual, audio, and textual game data available to human users, use one or more machine learning models to process the received data to determine the current game state, and determine the appropriate action to be performed based on the current game state. In some cases, this process can be performed on the user device 302. However, in many cases, the user device 302 may not have the computing resources required to perform two different game sessions and one or more machine learning models to simultaneously control the personalized agent game manipulation, or even if it is capable, it will not introduce a lag that negatively affects the user's game manipulation experience. In this way, the personalized agent service 306 can control the game manipulation of the personalized agent via the personalized agent game interface 326.

[0050] In some aspects, the personalized proxy service 306 may employ a personalized proxy game interface 326 operable to receive current game data (e.g., video data, audio data, text data, communications between players, tactile information, etc.) generated by an agent game instance 328 executed in a virtual machine. Figure 1A One or more machine learning models and / or other components of the personalized agent library discussed in the above process current game data received via the game interface 326 to interpret the current game state, determine personalized agent actions based on the current game state, and transmit control instructions to control the game manipulation of the instantiated personalized agent.

[0051] Although certain functions are described as being performed by specific devices or components of system 300, those skilled in the art will appreciate that other devices or components as part of system 300 may also be used. Figure 3 Other devices not shown in the figure may perform the described actions and functions without departing from the scope of the present disclosure. For example, although system 300 depicts both player game instance 308 and virtual machine game instance 328 as being executed on user device 302, in other aspects, both game instances may be executed on other devices. For example, the game instances may be executed on game platform 304 or on a game server ( Figure 3 Executed on (not shown).

[0052] Figure 4 An exemplary method 400 for creating a personalized agent game session and instantiating a personalized agent in an agent game session is depicted. In one example, the method 400 can be performed by a user device that performs a game session for a user's player character. In an alternative example, the method 400 can be performed using other devices, components, or cloud services described herein. The process begins at operation 402, where a request to invite a personalized agent to join a game session with a user is received. The request can be received via a contact or friend invitation interface. For example, a user can open their friend list and select one or more personalized agents to invite to the game. As described above, the request can be for a specific personalized agent and / or can include personalized agent characteristics. Upon receiving the request, the process continues to operation 404, where a request to allow the personalized agent to join the game session is generated and transmitted to the game service. At operation 404, details about the personalized agent (e.g., an identifier for a particular personalized agent, personalized agent characteristics, etc.), information about the user (e.g., a user identifier that can be used to associate the user with his or her personalized agent or personalized agent data, user character information, etc.), and / or details about the game (e.g., a game identifier, a world or server identifier representing the server on which the user is currently operating, etc.) can be aggregated and transmitted to a game service that manages the user's contact or friend list, user authorizations, user multi-player capabilities, etc., and / or a personalized agent game service.

[0053] In response to the transfer request, the process continues to operation 406, where the device executing method 406 receives authorization for the personalized agent game session. Although not shown, as described above, before receiving the authorization, a prompt requiring the user to obtain additional game permissions, or alternatively, additional agent permissions as described above, can be generated to perform additional game sessions. Once the authorization is received, the device executing method 400 has the data necessary to execute the additional instance of the game and connect the game to the online game service. The process then continues to operation 408, where the device executing method 400 creates a virtual environment in which the additional game session is executed. The virtual environment (e.g., a virtual machine, a container, etc.) allows the device executing method 400 to simultaneously execute two different game sessions for the same game (e.g., a game session for the user and a game session for the personalized agent).

[0054] In addition, as described above, the device executing method 400 can receive information about a personalized agent that allows the device to instantiate a personalized agent within a newly created game session in a virtual environment. The personalized agent information for instantiating a personalized agent can be received from a personalized agent service, a game platform, or a combination of the two. In one example, the personalized agent information can identify a personalized agent of a character in the game. In this way, a character in the game of a personalized agent can be selected at operation 410 to instantiate a personalized agent within a newly created game instance. Alternatively or additionally, the personalized agent data may include information about the characteristics of the agent, game data, components (e.g., one or more machine learning models) that control the game manipulation of the personalized agent, etc. In addition to or as a substitute for the character identification in the game of the personalized agent, this information can also be used to instantiate a personalized agent in the game.

[0055] As described above, since the game may not provide API access to allow the personalized agent to interact directly with the game, the personalized agent may interact with the game like a human player (e.g., interpreting the current game state through the visual and audio parts of the game and generating actions in response). However, the device performing method 400 may not have the computing resources to support two different game sessions and execute the machine learning model required to control the personalized agent game manipulation. Thus, in operation 412, a connection, such as with a personalized agent service, may be established. The connection can transmit the current game state to the personalized agent service (e.g., visual, audio, tactile, player communication, etc.), and in response, receive control signals to control the game manipulation of the personalized agent within the game instance executed in the virtual environment. The connection is persistent during the personalized agent's game manipulation, thereby allowing the personalized agent to continue playing with the user during the game session.

[0056] Figure 5An exemplary method 500 for determining whether a personalized agent game session can be established is depicted. In one example, the method 500 can be performed by a game platform. In an alternative example, the method 500 can be performed using other devices, components, or cloud services as described herein, or a combination thereof. The process begins at operation 502, where a request to invite a personalized agent to a game is received. For example, a request can be received from a user device that performs a game session for a user. In various aspects, details about the personalized agent (e.g., an identifier for a specific personalized agent, personalized agent characteristics, etc.), information about the user (e.g., a user identifier that can be used to associate a user with his personalized agent or personalized agent data, user character information, etc.) and / or details about the game (e.g., a game identifier, a world or server identifier representing a server on which the user is currently playing, etc.) can be included as parameters associated with the received request.

[0057] The process continues to operation 504, where a license repository is checked to see if the user requesting the proxy has the license required to establish a game session for the proxy. In various aspects, information about the user and the game associated with the request can be used to query the license repository to determine whether the user has an additional license for the game or whether the user has an unused proxy license for the game or game platform. Based on the results of the query, it is determined at operation 510 whether the user has the required license. If the user does not have the correct license, the process branches "no" to operation 508, where instructions prompting the user to obtain (e.g., purchase additional game licenses or proxy licenses) are transmitted to the requesting device. The process then returns to operation 504 (or alternatively 502), where the process continues until the required license is found in the repository.

[0058] If the user has the required game or agent license, the process branches "yes" from operation 506 to operation 510. At operation 510, the authorization required to establish an additional game session for the personalized agent is transmitted to the requesting device. As described above, the authorization can be used by the requesting device to establish a new game instance and connect to the game's service. The process continues to operation 512, where user and game information can be collected from one or more data stores managed by the game platform or the game itself. The information collected at operation 512 can be related to various aspects of the game required to personalize the agent for the game the user is playing and / or connect the personalized agent to the correct server or world to allow the agent to play with the user. At operation 506, the collected information is aggregated and sent to the personalized agent service and / or the requesting user device.

[0059] Fig. 6AAn exemplary method 600 for instantiating and controlling a personalized agent within a personalized agent game session is depicted. In one example, the method 500 may be performed by a gaming platform. In an alternative example, the method 500 may be performed using other devices, components, or cloud services, or a combination thereof, as described herein. The process begins at operation 602, where a request to instantiate a personalized agent in a gaming session is received. The request may include information about a pre-existing personalized agent, such as an identifier that identifies a specific agent. Alternatively or additionally, the request may include information about requested agent characteristics, characteristics of a user and / or a user's player character in a game, and / or information about a game in which the personalized agent is to be instantiated. The process continues to operation 604, where request parameters are analyzed to identify a specific personalized agent and / or characteristics requested or associated with the personalized agent. For example, the request parameters may be analyzed for specific characteristics to be included in the personalized agent. In addition, the method 600 may infer characteristics based on the request parameters. For example, information about a game (such as a game type or a game genre) may be used to identify agent characteristics associated with the game. For example, if the game is a role-playing game, the personalized selector 306 can identify agent features that are relevant or useful for role-playing. Similarly, if the game is a first-person shooter game, the personalized agent selector 322 can identify agent features related to first-person shooter games. Based on this analysis, specific personalized agents and / or features for creating personalized agents for the requested game session can be collected from the personalized agent data store associated with the user playing the game. At operation 606, aggregate information about the personalized agent is sent to the device hosting the game session of the personalized agent, with instructions for instantiating the personalized agent within the game session. For example, the instructions can be used to instantiate a character in the game associated with a past game session of the personalized agent character, instructions for creating a new character to be controlled by the personalized agent, etc. In one example, the personalized agent data and instructions for instantiating the personalized agent within the game session can be sent to the device executing the user's game session and / or the device executing the game session of the personalized agent.

[0060] As discussed, from the perspective of the game, there are now two different instances of the game operated by two different players. That is, the personalized agent is able to interact with the game without requiring the game to provide support for the agent to interact (e.g., without creating a specific API to allow the personalized agent to interact with the game). In this way, the multi-player functionality of the game can be utilized without modification to allow the user to invite their personalized agents to the game session when the user's friends are not online or otherwise unable to play the game with the user. However, because the game does not need to expose an API to allow the agent to interact, the personalized agent is able to participate in the game and interact with it like a normal human player. That is, the personalized agent is operable to receive visual, audio, and textual game data available to a human user, process the received data using one or more machine learning models to determine a current game state, and determine appropriate actions to perform based on the current game state. Thus, at operation 608, a device hosting the personalized agent (e.g., from Figure 3 The communication session is used to receive game state information in the form of audio data, visual data, tactile data, text data, etc. In addition, the communication session established at operation 608 can be used by the device performing method 600 to transmit control instructions to control the game manipulation of the personalized agent.

[0061] When the communication session is established, the process continues to operation 610, where the device performing method 600 receives game data (e.g., visual, audio, tactile data) generated by the game session of the personalized agent. The game data received at operation 610 can be the same game data available to human players. That is, the game data does not need to include API access to game data that is unavailable to human players. At operation 612, one or more machine learning models are used to analyze the received game data. For example, the received game data can be provided to one or more basic machine learning models, object recognition models, speech recognition models, natural language understanding models, etc., in order to process the current game state. At operation 614, the output of one or more machine learning models (alone or based on the further processing of other components of the personalized agent library 107 described as a part of Figure 1) can be used to generate instructions to control the interaction of the personalized agent during the game manipulation in response to the current game state. The action, for example, includes where to move the character of the personalized agent, what ability or action the personalized agent character should perform, the list of accessing the personalized agent character, etc. That is, any actions available for execution by the player in the game may be determined at operation 614 and transmitted to the device hosting the gaming session of the personalized agent, thereby causing the instance of the personalized agent to execute the action in the game. In doing so, the personalized agent is able to interact and participate in the game without the need to specifically develop the game to support the personalized agent. In this way, the personalized agent developed over time through the user's continued game manipulation can be adopted by the user in the future to play with one or more personalized agents in the same or different games.

[0062] Flow continues to operation 616, where it is determined whether the game is still in session. If so, then the flow branches "no" to operation 610, where game data continues to be received and processed by the device executing method 600 to allow for continued game manipulation between the personalized agent and the user. However, if the game has ended, then the flow branches "yes" to operation 618, where the communication session ends and control of the personalized agent ceases.

[0063] Figure 6BAn example of a method 650 for utilizing computer vision to enable a personalized agent to interact with a user during game play is depicted. The process begins at operation 652, where an interaction is received from a user. The interaction may be a voice or text interaction. For example, a user may ask, "Does this object look like a house?" via a voice interface or via a chat interface. Alternatively, the interaction may be received via a user action rather than a user communication. Upon receiving the interaction, the process continues to operation 654, where the interaction is analyzed to determine a user request and / or user intent associated with the action. For example, if the interaction is a user communication, a speech recognition and / or natural language understanding model may be used to process the communication to determine the request made by the user and identify the intent associated with the request. If the interaction is a user action, other techniques such as computer vision techniques, event recording techniques, etc. may be employed to determine the intent behind the action or whether the action implies a user request.

[0064] Upon determining the request and / or intent, the game environment may be visually processed at operation 656 to determine a response to the request. That is, instead of accessing game data, the robot may visually inspect the game environment using computer vision techniques to determine the state of the game as a human user would. For example, if the user request is "Does this look like a house?", the robot may analyze objects in its surrounding environment to identify objects that look like houses using computer vision and object detection techniques, as opposed to, for example, accessing game or state data to determine whether any of the nearby objects are marked or otherwise identified as houses. Based on this analysis, the process continues to operation 658, where the robot generates a response to the request based on information determined from visually analyzing the game environment.

[0065] Although the specific examples described herein relate to utilizing agents in a gaming environment, those skilled in the art will appreciate that the techniques described can be applied to generating and using agents in other types of environments, such as enterprise environments. For example, a personalized agent can be generated to help a user perform tasks in an enterprise environment, or use any other type of application.

[0066] Figure 7is a block diagram illustrating the physical components (e.g., hardware) of a computing device 700 with which aspects of the present disclosure may be practiced. The computing device components described below may be applicable to the computing devices described above. In a basic configuration, the computing device 700 may include at least one processing unit 702 and a system memory 704. Depending on the configuration and type of the computing device, the system memory 704 may include, but is not limited to, volatile storage devices (e.g., random access memory), non-volatile storage devices (e.g., read-only memory), flash memory, or any combination of such memories. The system memory 704 may include an operating system 705 and one or more program tools 706 suitable for performing various aspects disclosed herein. The operating system 705 may, for example, be suitable for controlling the operation of the computing device 700. In addition, aspects of the present disclosure may be practiced in conjunction with a graphics library, other operating systems, or any other application, and is not limited to any particular application or system. The basic configuration in Figure 7 708. The computing device 700 may have additional features or functionality. For example, the computing device 700 may also include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage devices may be used to store data or other data. Figure 7 , which is illustrated by a removable storage device 709 and a non-removable storage device 710.

[0067] As described above, a plurality of program tools and data files may be stored in system memory 704. When executed on at least one processing unit 702, program tools 706 (e.g., applications 720) may perform processes including, but not limited to, various aspects as described herein. Applications 720 include personalized agent generators 730, machine learning models 732, game sessions 734, personalized agent controllers 736, and instructions for performing various processes disclosed herein. Other program tools that may be used according to various aspects of the present disclosure may include email and contact applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided applications, and the like.

[0068] In addition, aspects of the present disclosure may be practiced in electronic circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing a microprocessor, or a single chip containing electronic components or a microprocessor. For example, aspects of the present disclosure may be practiced via a system on a chip (SOC), wherein Figure 7Each or more components shown in can be integrated into a single integrated circuit. Such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") into a chip substrate as a single integrated circuit. When operated via a SOC, the functions described herein regarding the ability to switch protocols for clients can be operated via dedicated logic integrated with other components of the computing device 700 on a single integrated circuit (chip). Various aspects of the present disclosure may also be practiced using other technologies capable of performing logical operations (such as AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, various aspects of the present disclosure may be practiced within a general-purpose computer or any other circuit or system.

[0069] The computing device 700 may also have one or more input devices 712, such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or sliding input device, etc. Output devices 714 may also be included, such as a display, a speaker, a printer, etc. The aforementioned devices are examples, and other devices may be used. The computing device 700 may include one or more communication connections 716 that allow communication with other computing devices 750. Examples of communication connections 716 include, but are not limited to, radio frequency (RF) transmitters, receivers, and / or transceiver circuits; universal serial bus (USB), parallel and / or serial ports.

[0070] The term "computer-readable medium" as used herein may include computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, or program tools). System memory 704, removable storage device 709, and non-removable storage device 710 are all examples of computer storage media (e.g., memory storage). Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other article that can be used to store information and can be accessed by computing device 700. Any such computer storage media may be part of computing device 700. Computer storage media do not include carrier waves or other propagated or modulated data signals.

[0071] Communication media may be embodied by computer readable instructions, data structures, program tools, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" may describe a signal that has one or more characteristics set or changed in a manner that encodes information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0072] Fig. 8A and Figure 8B A computing device or mobile computing device 800 is shown, such as a mobile phone, a smart phone, a wearable computer (such as a smart watch), a tablet computer, a laptop computer, etc., in which aspects of the present disclosure can be practiced. In some aspects, a client used by a user (e.g., client device 102 as shown in system 100 in FIG. 1 ) can be a mobile computing device. Fig. 8A , shows one aspect of a mobile computing device 800 for implementing various aspects. In a basic configuration, the mobile computing device 800 is a handheld computer with proxy input elements and output elements. The mobile computing device 800 typically includes a display 805 and one or more input buttons 810 that allow a user to input information into the mobile computing device 800. The display 805 of the mobile computing device 800 can also be used as an input device (e.g., a touch screen display). If included as an optional input element, the side input element 815 allows further user input. The side input element 815 can be a rotary switch, a button, or any other type of manual input element. In alternative aspects, the mobile computing device 800 can incorporate more or fewer input elements. For example, in some aspects, the display 805 may not be a touch screen. In yet another alternative aspect, the mobile computing device 800 is a portable telephone system, such as a cellular phone. The mobile computing device 800 can also include an optional keypad 835. The optional keypad 835 can be a physical keypad or a "soft" keypad generated on the touch screen display. In various aspects, output elements include a display 805 for showing a graphical user interface (GUI), a visual indicator 820 (e.g., a light emitting diode), and / or an audio transducer 825 (e.g., a speaker). In some aspects, the mobile computing device 800 includes a vibration transducer for providing tactile feedback to the user. In yet another aspect, the mobile computing device 800 incorporates input and / or output ports for sending or receiving signals to or from external devices, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., an HDMI port).

[0073] Figure 8B1 is a block diagram illustrating an architecture of one aspect of a computing device, a server (e.g., an application server 104, an incident data server 106, and an incident correlator 110, as shown in FIG. 1 ), a mobile computing device, and the like. That is, a mobile computing device 800 can be incorporated into a system 802 (e.g., a system architecture) to implement some aspects. The system 802 can be implemented as a "smart phone" capable of running one or more applications (e.g., a browser, email, calendar, contact manager, messaging client, game, and media client / player). In some aspects, the system 802 is integrated into a computing device, such as an integrated digital assistant (PDA) and a wireless phone.

[0074] One or more application programs 866 may be loaded into the memory 862 and run on or in association with the operating system 864. Examples of application programs include a phone dialer program, an email program, a information management (PIM) program, a word processing program, a spreadsheet program, an Internet browser program, a messaging program, and the like. The system 802 also includes a non-volatile storage area 868 within the memory 862. The non-volatile storage area 868 may be used to store persistent information that should not be lost when the system 802 loses power. The application programs 866 may use and store information in the non-volatile storage area 868, such as emails or other messages used by the email application, and the like. A synchronization application (not shown) also resides on the system 802 and is programmed to interact with a corresponding synchronization application residing on a host computer to keep the information stored in the non-volatile storage area 868 synchronized with the corresponding information stored at the host computer. It should be understood that other applications may be loaded into the memory 862 and run on the mobile computing device 800 described herein.

[0075] The system 802 has a power source 870, which can be implemented as one or more batteries. The power source 870 can also include an external power source, such as an AC adapter or a powered docking cradle that replenishes or recharges the batteries.

[0076] The system 802 may also include a radio interface layer 872 that performs the function of sending and receiving radio frequency communications. The radio interface layer 872 facilitates wireless connectivity between the system 802 and the "outside world" via a communications carrier or service provider. Transmissions to and from the radio interface layer 872 are performed under the control of the operating system 864. In other words, communications received by the radio interface layer 872 may be propagated to the application 866 via the operating system 864, and vice versa.

[0077] Visual indicator 820 (e.g., LED) can be used to provide visual notification, and / or audio interface 874 can be used to generate audible notification via audio transducer 825. In the configuration shown, visual indicator 820 is a light emitting diode (LED), and audio transducer 825 is a speaker. These devices can be directly coupled to power supply 870 so that when it is activated, they remain turned on for the duration indicated by the notification mechanism, even if processor 860 and other components may be turned off to save battery power. LED can be programmed to remain on indefinitely until the user takes action to indicate the power-on state of the device. Audio interface 874 is used to provide audible signals to the user and receive audible signals from the user. For example, in addition to being coupled to audio transducer 825, audio interface 874 can also be coupled to a microphone to receive audible input, such as promoting telephone conversations. According to aspects of the present disclosure, microphone can also be used as an audio sensor to facilitate the control of notifications, as described below. System 802 can also include video interface 876, which enables the operation of the device connected to peripheral device port 830 to record still images, video streams, etc.

[0078] The mobile computing device 800 implementing the system 802 may have additional features or functionality. For example, the mobile computing device 800 may also include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage devices may be used to store data or other data. Figure 8B , shown by non-volatile storage area 868.

[0079] Data / information generated or captured by the mobile computing device 800 and stored via the system 802 may be stored locally on the mobile computing device 800, as described above, or the data may be stored on any number of storage media that may be accessed by the device via the radio interface layer 872 or via a wired connection between the mobile computing device 800 and a separate computing device associated with the mobile computing device 800 (e.g., a server computer in a distributed computing network such as the Internet). It should be understood that such data / information may be accessed by the mobile computing device 800 via the radio interface layer 872 or via a distributed computing network. Similarly, such data / information may be readily transferred between computing devices for storage and use in accordance with well-known data / information transfer and storage components, including electronic mail and collaborative data / information sharing systems.

[0080] In some examples, a system for controlling game play of a personalized agent is disclosed, the system comprising: at least one processor; and a memory encoding computer executable instructions that, when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising: receiving a request from a user device to instantiate a personalized agent in a personalized agent game play session, wherein the personalized agent uses personalized agent data, and wherein the personalized agent data is generated based on operations comprising: instantiating an agent in a game play session with a user for a first game, wherein the instantiated agent is operable to interact with a user playing the first game based on one or more machine learning models, the one or more machine learning models being trained to play the game; receiving user interactions by the agent during game play; generating an agent response to the user interaction via the one or more machine learning models; instructing the agent to perform the agent response; receiving feedback from the user on the agent response; and generating agent personalized data based on the agent response and the user feedback; determining personalized agent data for instantiating the agent based on the request; sending the personalized agent data to the user device; establishing a communication session with the user device; receiving game data from the user device; analyzing the game data using the one or more machine learning models to determine one or more actions to be performed by the personalized agent; and sending the one or more actions to the user device.

[0081] In some examples, the one or more machine learning models include: a base model; a language model; a computer vision model; or a speech model.

[0082] In some examples, the video game data includes one or more of: visual data; audio data; haptic feedback data; textual data; or data exposed via an API.

[0083] In some examples, analyzing the game data includes: providing at least a portion of the video game data to a machine learning model trained to analyze the portion of the game data; and determining a current game state based on an output of the machine learning model.

[0084] In some examples, one or more actions are determined based on the current game state.

[0085] In some examples, the request includes parameter data related to at least one of: an identifier for a particular personalized agent; data related to one or more desired characteristics of the personalized agent; game data; or user data.

[0086] In some examples, the game data also includes at least one of: a game identifier; information about the genre of the game; or a server or world on which the user character is playing.

[0087] In some examples, the user data includes at least one of: data detailing characteristics of a user persona; or data detailing a user's preference manipulation strategy.

[0088] In some examples, a method for instantiating a personalized agent in a game instance is provided, the method comprising: executing a first instance of a game played by a user on a user device; receiving a request for the personalized agent to join the game; sending the request to a game service; in response to receiving the send request, receiving authorization for the personalized agent game instance; executing a second instance of the game for the personalized agent on the user device; receiving personalized agent data from the personalized agent service; and instantiating the personalized agent in the second instance of the game.

[0089] In some examples, the request is received via selection of a personalized agent from a friends list associated with the user.

[0090] In some examples, the selection is for a specific personalized agent.

[0091] In some examples, receiving the selection also includes receiving one or more characteristics for the personalized agent.

[0092] In some examples, authorization for a personalized agent game instance is based on an agent license that is different than a game license for the game.

[0093] In some examples, the method further includes: establishing a communication session with the personalized agent service; sending game data to the personalized agent service via the communication session, wherein the game data includes at least one of the following: visual data, audio data, or tactile data; and in response to sending the game data, receiving one or more commands to control the game manipulation of the personalized agent.

[0094] In some examples, a second instance of the game is executed within a virtual environment on the user device.

[0095] In some examples, a method for determining whether a user has permission to instantiate a personalized agent includes: receiving a request from a user device to invite the personalized agent to play a game with the user; determining whether the user has permission for a second instance of the game; when the user has permission for the second instance of the game, sending authorization information to the user device, wherein the authorization information is operable to allow the user device to instantiate the second instance of the game; and sending a second request to the personalized agent service to cause instantiation of the personalized agent in the second instance of the game.

[0096] In some examples, determining whether the user has a license for the second instance includes querying a license repository to determine whether the user has a second license for the game.

[0097] In some examples, determining whether the user has permission for the second instance includes querying a permission repository to determine whether the user has a proxy permission for the game.

[0098] In some examples, the proxy license allows the user to execute a second instance of the game for use by the proxy, and wherein the proxy license does not allow the user to execute the second instance of the game for use by another human player.

[0099] In some examples, the proxy license is obtained separately from the general license for the game.

[0100] The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the disclosure claimed for protection in any way. The disclosure claimed for protection should not be interpreted as being limited to any aspect or detail provided in the application, for example. Whether combined or shown and described separately, various features (agent structures and methods) are intended to be selectively included or omitted to produce aspects with a specific feature set. After the description and illustration of the application have been provided, those skilled in the art can envision changes, modifications and alternative aspects within the spirit of the broader aspects of the general inventive concept embodied in the application, which do not depart from the broader scope of the disclosure claimed for protection.

Claims

1. A system comprising: at least one processor; as well as a memory encoding computer executable instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising: Receiving a request from a user device to instantiate a personalized agent in a personalized agent game play session, wherein the personalized agent uses personalized agent data, and wherein the personalized agent data is generated based on operations including: instantiating an agent in a game play session with a user for a first game, wherein the instantiated agent is operable to interact with the user playing the first game based on one or more machine learning models, the one or more machine learning models being trained to play the game; receiving, by the agent, user interactions during gameplay; generating, via the one or more machine learning models, an agent response to the user interaction; instructing the agent to execute the agent response; receiving feedback from the user on the agent response; and generating agent personalization data based on the agent response and user feedback; Based on the request, determining personalized agent data for instantiating an agent; Sending the personalized agent data to the user equipment; establishing a communication session with the user equipment; receiving game data from the user device; analyzing the game data using the one or more machine learning models to determine one or more actions to be performed by the personalized agent; and The one or more actions are sent to the user equipment.

2. The system of claim 1, wherein the one or more machine learning models include: Basic model; Language model; Computer vision models; or Voice model.

3. The system of claim 2, wherein the video game data comprises one or more of: Visual data; Audio data; Haptic feedback data; text data; or Data exposed via API.

4. The system of claim 3, wherein analyzing the game data comprises: providing at least a portion of the video game data to a machine learning model trained to analyze the portion of the game data; as well as Based on the output of the machine learning model, determine the current game state.

5. The system of claim 4, wherein the one or more actions are determined based on the current game state.

6. A method comprising: executing a first instance of a game played by a user on a user device; receiving a request for a personalized agent to join the game; Sending the request to the game service; In response to receiving the request to send the request, receiving authorization for the personalized agent game instance; executing a second instance of the game for the personalized agent on the user device; receiving personalized agent data from a personalized agent service; as well as The personalized agent is instantiated in the second instance of the game.

7. The method of claim 6, wherein the request is received via selection of the personalized agent from a buddy list associated with the user.

8. The method according to claim 6, further comprising: establishing a communication session with the personalized proxy service; sending game data to the personalized agent service via the communication session, wherein the game data includes at least one of: visual data, audio data, or tactile data; as well as In response to sending the game data, one or more commands are received to control game play of the personalized agent.

9. The method of claim 6, wherein the second instance of the game is executed within a virtual environment on the user device.

10. The method of claim 6, wherein the authorization for a personalized agent game instance is based on an agent license that is different than a game license for the game.

11. A method comprising: receiving a request from a user device to invite the personalized agent to play a game with the user; determining whether the user has a license for a second instance of the game; sending authorization information to the user device when the user has the license for the second instance of the game, wherein the authorization information is operable to allow the user device to instantiate the second instance of the game; as well as A second request is sent to a personalization agent service to cause instantiation of the personalization agent in the second instance of the game.

12. The method of claim 11, wherein determining whether the user has permission for the second instance comprises: A license repository is queried to determine whether the user has a second license for the game.

13. The method of claim 11, wherein determining whether the user has permission for the second instance comprises: A license repository is queried to determine whether the user has an agent license for the game.

14. The method of claim 13, wherein the proxy license permits the user to execute the second instance of the game for use by a proxy, and wherein the proxy license does not permit the user to execute the second instance of the game for use by another human player.

15. The method of claim 11, wherein the proxy license is obtained separately from a general license for the game.

Images