Importing agent anthropomorphic data for instantiating anthropomorphic agents within a user game session

Abstract

Aspects of the present disclosure relate to an anthropomorphic agent service that generates and evolves customizable agents that can be instantiated within a game to play with a user. A machine learning model is trained to control the interaction of agents during gameplay with the game environment and the user. A user may request that an anthropomorphic agent participate in the user's gameplay session. The user device transmits a request for the anthropomorphic agent to the game platform. The game platform determines whether the user has a license to execute a second instance of the game. If the user has a license to execute a second instance of the game, the second instance of the game may be executed on the user device. Information received from the anthropomorphic agent service is used to instantiate the anthropomorphic agent within the second instance of the game.

Description

Background Art

[0001] The gaming industry occupies a large part of the technology industrial sector. In a future where technology anticipates an increasingly connected future, games are a central element of that connected future. Artificial intelligence has always been a major element throughout the gaming industry, but as AI development continues to progress, it becomes more difficult for the gaming industry to incorporate that progress into games. For example, since most advanced AI systems require a lot of custom domain knowledge, it is difficult for game development studios to incubate AI.

Summary of the Invention

Means for Solving the Problems

[0002] Overview Aspects disclosed herein are provided in view of these and other general considerations. Additionally, although relatively specific problems may be discussed, it should be understood that the examples should not be limited to solving the specific problems made apparent in the background or elsewhere of the present disclosure.

[0003] Aspects of the present disclosure relate to an anthropomorphic agent service that generates and evolves customized agents that can be instantiated within a game for a user to play with. A machine learning model is trained to control the interaction between agents during gameplay and the game environment and the user. As the user continues to play with the agent, one or more machine learning models complement the user's preferred play style, incorporate the user's preferred strategies, and develop a gameplay style for the agent generally customized for interaction with the user. Agent anthropomorphic data generated during gameplay is stored by the service, thereby enabling the user to import the agent into different games, thereby creating a constant gameplay companion that the user can play with across a variety of different games without the need for those different games to include technology capable of developing AI-controlled agents.

[0004] In a further aspect, a user device is disclosed that enables a user to request that an anthropomorphic agent participate in the user's game play session. The user device sends a request for the anthropomorphic agent to the game platform. The game platform determines whether the user has a license to execute a second instance of the game. If the user has a license to execute a second instance of the game, the second instance of the game can be executed on the user device. Information received from the anthropomorphic agent service is used to instantiate the anthropomorphic agent within the second instance of the game. A communication session is established between the device executing the second instance of the game and the anthropomorphic agent service. The anthropomorphic agent service uses one or more machine learning models to analyze game data to determine the current game state. Based on the current game state, one or more actions are determined to control the game play of the anthropomorphic agent.

[0005] This summary is provided to introduce in a simplified form a series of concepts that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additional aspects, features, and / or advantages of the examples will be described in part in the following description, become apparent in part from the description, or may be learned by practice of the disclosure.

[0006] Brief Description of the Drawings Non-limiting and non-exhaustive examples are described with respect to the following drawings.

Brief Description of the Drawings

[0007]

Fig. 1A

Fig. 1B

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6A

Fig. 6B

Fig. 7

Fig. 8A

Fig. 8B

MODE FOR CARRYING OUT THE INVENTION

[0008] Detailed Description This specification forms a part of it and describes various aspects of the present disclosure more fully hereinafter with reference to the accompanying drawings that illustrate specific example embodiments. However, the various aspects of the present disclosure can be implemented in many different ways and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the aspects to those skilled in the art. The embodiments practiced may be in the form of a method, system, or apparatus. Thus, the aspects can take the form of a hardware implementation, an entirely software implementation, or an implementation combining software and hardware aspects. Accordingly, the following detailed description should not be taken in a limiting sense.

[0009] Aspects of the present disclosure provide systems and methods that utilize machine learning techniques to provide anthropomorphic agents or bots that can be used within a game or other type of environment. Among several examples, in particular, the aspects disclosed herein relate to using a reinforcement learning agent that can be trained by computer vision without relying on deep game hooks, an agent that can understand text by writing and speaking, has a defined knowledge system, and is operable to write its own code, training the agent using models (such as base models, language models, computer vision models, speech models, video models, audio models, multimodal machine learning models, etc.), providing an agent that interprets state information using computer vision and audio cues, providing an agent that receives user commands based on computer vision and audio cues, and providing a system in which user feedback is used to improve future interactions with the agent across various games and applications.

[0010] In an example, the generative multimodal machine learning model processing processes user input and generates a multimodal output. For example, a conversation agent according to the aspects described herein can receive user input and thereby use the generative multimodal machine learning model to process the user input to generate a multimodal output. The multimodal output can include, among several examples, in particular natural language output and / or program output. The multimodal output can be processed and used to affect the state of the relevant application. For example, at least a part of the multimodal output may be executed, or it can be used to call an application programming interface (API) of the application. The generative multimodal machine learning model (also generally referred to herein as the multimodal machine learning model) used according to the aspects described herein can be a generative transformer model in some examples. In some examples, explicit and / or implicit feedback can be processed to improve the performance of the multimodal machine learning model.

[0011] In an example, the user input and / or the model output are multimodal, and as used herein, multimodal can include one or more types of content. Examples of content include, but are not limited to, in particular speech or written language (which may also be referred to herein as "natural language output"), code (which may also be referred to herein as "program output"), images, videos, audio, gestures, visual features, intonation, contour features, poses, styles, fonts, and / or transitions. Thus, compared to a machine learning model that processes natural language input and generates natural language output, the aspects of the present disclosure can process input having any of a wide variety of types of content and generate an output.

[0012] By doing so, the systems and methods disclosed herein support an anthropomorphic agent that learns about who the user (also synonymously referred to herein as the "player") is, how the user speaks, what the user's strategy is, and how the user plays. The anthropomorphic agent maintains a "memory" of past interactions with the user and can function as a constant companion to the user when engaging in different experiences, such as when the user plays different games, plays in different game modes, etc. To that end, the aspects disclosed herein are operable to store meta-classifications (e.g., sentiment analysis, intent analysis, etc.) related to personal data about the user, conditional upon the user having given permission.

[0013] Figure 1 shows an overview of an example of a system 100 for generating and using a user anthropomorphic agent in a game system. As shown in Figure 1, the user device 102 communicates with a cloud service 104 that hosts an instantiation of a game service 106 (or other type of application) and an agent 108 capable of interacting with the game. The game device can be a console game system, a mobile device, a smartphone, a personal computer, or any other type of device capable of locally executing a game or accessing a game hosted on a server. In one example, the game associated with the game service 106 can be directly hosted by the cloud service 104. In an alternative example, the user device may host and execute the game locally, in which case the game service 106 can serve as an interface that facilitates communication between one or more instantiated agents 108 and the game. The anthropomorphic agent library 107 can store and execute components for one or more agents related to the user of the user device 102. The components of the anthropomorphic agent library 107 can be used to control the instantiated agent 108.

[0014] In an example, one or more agents from the anthropomorphic agent library 107 interact with the game via the instantiated agent 108 based on text communications, voice commands, and / or player actions received from the user device 102. That is, the system 100 supports interaction with the agent as if the agent were another human player, or as a player would normally interact with an NPC within a conventional game. By doing so, one or more agents hosted by the agent library 108 can operate to interact with various games that a user plays without requiring changes to the game. That is, the system 100 can operate to cooperate with the game without requiring the game to be specially developed to support the agent (e.g., the agent does not require API access, and the game does not need to be developed with specific code to interact with the agent). By doing so, the system 100 provides an extensible solution that enables a user to play with customized agents across a wide variety of different games. That is, the game state is a game by the game service 106 that communicates between one or more instantiated agents 108 using voice and visual interaction and / or a public API. By doing so, the instantiated agent 108 can interact with the game in the same way as the user (e.g., by interpreting video, audio, and / or tactile feedback from the game) and / or in the same way as an NPC within the game (e.g., by the public API). Similarly, one or more agents can interact with a user playing a game using the game device 102 in the same way as another player or an NPC that interacts with the user. That is, one or more agents can receive visual, text, or voice input from the user playing the game by the game service 106 and / or game information (e.g., the current game state, the inventory of the NPC, the abilities of the NPC, etc.) by the public API.In this way, the system 100 enables a user playing a game on the user device 102 to interact with one or more agents such that the user can interact with any other player on the NPC. However, the instantiated agent 108 can be personalized according to the user based on past interactions with the user that occurred in both the current game and other games. To facilitate this type of user interaction, one or more instantiated agents 108 can employ computer vision, speech recognition, or other known techniques when processing the interaction with the user in order to interpret the commands received from the user and / or generate response actions based on the user's actions.

[0015] For example, consider a user who plays a first-person shooting with a personalized agent. The user may say "Cover the right side". A speech recognition model, which may be one of the models 118, can be utilized to interpret the voice received from the user in a manner that can be understood by the agent. Accordingly, the agent 108 can take a position on the right side of the user in response to the user's voice command or execute an action to cover the right side of the user. As yet another example, consider an action role-playing game where two objectives must be defended simultaneously. The agent playing with the user can use computer vision to analyze the current view of the game and determine that there are two objectives (e.g., identify two objectives on a map, interpret a display quest log indicating that two objectives must be defended, etc.). Similarly, computer vision can be used to identify the user's player character and determine that the player character is heading towards a first objective. Based on the feedback from the computer vision model, the agent can move its proxy character towards the second objective and command it to defend it.

[0016] From these examples, those skilled in the art will understand that by adopting speech recognition, computer vision techniques, etc., one or more agents from the agent library can interact with the user in the same way as other users do during cooperative gameplay. By doing so, the instantiated agent 108 may not have API access or program access to the game to interact with the user, so one or more agents from the agent library can be generated separately from a specific game. Alternatively, in the case where the instantiated agent "depends" and for instantiated agents such as NPCs in the game, the instantiated agent 108 may have API access or program access to the game. In such a situation, the agent can interact with the game state and the user based on the API access. Since the system 100 provides a solution that can implement agents separately from individual games, one or more agents in the agent library can be anthropomorphized to interact with a specific user. This anthropomorphization can be maintained across various games. That is, the agent can learn details about the user over time, such as the user's strengths and weaknesses, play style, communication pattern, preferred strategies, etc., and can appropriately respond to the user across various games.

[0017] The anthropomorphization of the agent can be generated by the feedback collection engine 110 and updated over time. The feedback collection engine 110 receives feedback occurring within the game from the user and / or the instantiated agent 108. The feedback collected may include the user's play style, the user's communication, the user's interaction with the game, the user's interaction with other players, the user's interaction with other agents, the results of actions performed by the instantiated agent 108 within the game, information related to the interaction between the actions of the players within the game and the actions of the instantiated agent 108, or any kind of information generated by the user device 102 when the user plays the game. To comply with considerations regarding the user's privacy, the information can only be collected by the feedback collection engine 110 when permission is obtained from the user. The user can choose to participate or not participate in the collection at any time. The data collected can be implicit data, such as data based on the user's normal interaction with the game, or explicit data, such as specific commands provided by the user to the system. An example of a specific command could be for the user to instruct the agent to be addressed by a specific character name. The data collected by the feedback collection engine 110 can be provided to the prompt generator 112.

[0018] The prompt generator 112 can generate a prompt for use in anthropomorphizing one or more agents of the agent library 108 using the data collected by the feedback collection engine 110. That is, the prompt generator 112 interprets the collected feedback data to generate instructions that one or more agents can execute to perform actions by the agents. The prompt generator is operable to generate new prompts or instructions based on the collected feedback or to modify existing prompts based on newly collected feedback. For example, if a user first plays a game as a frontline attacker, the prompt generator 112 can generate instructions to implement a supportive play style, such as a long-range attacker or a support character, for the instantiated agent 108. If the user transitions their play style to one of the long-range damage dealers, the prompt generator 112 can identify that change by the feedback data and adjust the instantiated agent to adapt to the player's new style (e.g., switch to a frontline attacker to remove confrontation or adversarial situations from the player's character). The instructions generated by the prompt generator are provided to the cloud service 104 to be stored by the cloud service 104 as part of the agent library 108, thereby storing meta-classifications (e.g., sentiment analysis, intent analysis, etc.) related to a particular user. By doing so, the instructions generated based on the user's play style or preferences in the first game can be incorporated by the agent not only in the first game but also in other games the user plays. That is, using the instructions generated by the prompt generator 112, the cloud service 104 can instantiate agents across a variety of different games that have already been anthropomorphized for a particular user based on the user's past interactions with the instantiated agents 108, regardless of whether the user is playing the same game or different games.The aspects described in this specification are described as a separate prompt generator 112 generating commands to control the instantiated agent 108. However, in alternative aspects, the commands may be directly generated by one or more machine learning models employed by the agent library 107, or may be directly generated by a combination of various different components disclosed herein.

[0019] However, in some scenarios, there may be problems encountered during the training process of various machine learning models that can be utilized to generate anthropomorphic agents. For example, the training session may fail due to data errors or processing errors. In yet another example, the training process may fail due to user errors. For example, a user starting a new game may initially play the game incorrectly. The user's incorrect actions or experiences may train the anthropomorphic agent to play in a way that negatively impacts the gameplay. Therefore, the system 100 may include a process for rolling back or resetting one or more machine learning models (or any of the other components disclosed herein) to correct errors that may occur during the training of one or more machine learning models, or other errors that may generally occur when an anthropomorphic agent is developed. For example, the system 100 can periodically maintain snapshots of various machine learning models (or other components) that save the state of the components at the time of the snapshot. Thereby, the system 100 can roll back all components, a subset of components, or a specific component in response to detecting a training error in the future. As the anthropomorphic agent evolves over time, multiple different snapshots representing its state can be stored as a store, thereby providing the system 100 (or the user) with options for determining a state suitable for rollback when an error is encountered.

[0020] The anthropomorphic agent library 107 may also include a fine-tuning engine 114 and one or more models 116. The fine-tuning engine 114 and the models 116 are operable to interact with user actions by the user device 102, the instantiated agent 108, and the game training model 124 to process feedback data received from various sources. For use with the examples disclosed herein, any number of different models can be employed individually or in combination. For example, a base model, a language model, a computer vision model, a speech model, a video model, and / or an audio model may be employed by the system 100. As used herein, a base model is a model trained based on a wide range of data adaptable to a wide range of tasks (e.g., a model capable of processing various different tasks or styles).

[0021] One or more models 116 can process video, audio, and / or text data received from the user or generated by the game during gameplay to interpret the user's commands and / or derive the user's intent based on the user's communication or in-game actions. The output from one or more models 116 is provided to the fine-tuning model 114, and the fine-tuning model 114 can use its output to modify the prompt generated by the prompt generator 112 to further anthropomorphize the commands generated for the user based on the agent's past interactions with the user. The anthropomorphic agent library 107 may also include a game data (heritage) 122 component operable to store information about various different games. The game data (heritage) 122 component stores information about the game, the game's heritage, story elements, available capabilities, and / or items, etc., that can be utilized by the agent by other components (e.g., model 118, fine-tuning engine 114, prompt generator 112, etc.) to generate instructions for controlling the instantiated agent 108 according to the theme, story, requirements, etc. of the game.

[0022] The various components described herein can be utilized to interpret the current game state based on data received from the game (e.g., visual data, audio data, tactile feedback data, data that the game exposes via an API, etc.). Further, the components disclosed herein are operable to generate instructions for controlling the actions of a humanoid agent within the game based on the current game state. For example, the components disclosed herein can be operable to generate instructions for controlling the interaction of the humanoid agent with the game in a manner similar to how a human interacts with the game (e.g., by providing specific controller instructions, keyboard instructions, or any other type of interface having supported game play controls). Alternatively or in addition, the various components may generate code that controls how the humanoid agent interacts within the game. For example, the code may be executed by the humanoid agent, and such execution causes the humanoid agent to perform specific actions within the game.

[0023] The humanoid agent library 107 can also include an agent memory component 120. The agent memory component can be used to store the humanoid data generated by the various other components described herein, as well as the play styles, techniques, and interactions that the agent has learned from past interactions with the user. The agent memory 120 can provide additional input to the prompt generator 112 that can be used to determine the actions of the instantiated agent during game play.

[0024] So far, the described components of system 100 have focused on the creation of anthropomorphic agents, the continuous evolution of anthropomorphic agents through continuous interaction with users, and the instantiation of anthropomorphic agents in a user's game session. The anthropomorphization of agents for a particular user provides many in-game advantages, but the instantiated agents also require an understanding of how to interact with the game and how to play the game. In one aspect, the components of anthropomorphic agent library 107 are operable to learn and refine the game play of agents based on a session with a user. However, when a user plays a new game, the time required for the agent to learn the mechanics of the game play to be a useful companion may not be achievable with just the game play with the user. To address this problem, system 100 also includes a game play training service 124 that includes a game library 126 and a game play machine learning model 128. In various aspects, the game library includes any number of games supported by cloud service 104 and / or user device 102. Game play training service 124 is operable to execute sessions of various games stored within game library 126 and to instantiate agents within the executed games. Game play machine learning model 128 is operable to receive as input data from the executed games and the actions of the agents taken within the games, and in response, generate control signals that direct the game play of the agents within the games. Through the use of reinforcement learning, the game play machine learning model is operable to deepen the understanding of the mechanics of game play for both a particular game and game genre. By doing so, game play training service 124 provides a mechanism by which agents can be trained to play a particular game or a particular type of game without the need for user interaction.The anthropomorphic agent library 107 is operable to receive (or interact with) a trained model that can be stored as part of the agent memory from the game play training service 124 and control an agent 108 instantiated using those models along with other anthropomorphic components. This significantly improves the user experience of interacting with agents within the game, as it eliminates the need for the user to spend time training agents in the specific mechanics of the game. Additionally, by importing or interacting with the trained game play machine learning model 128 provided by the game play training service 124, the anthropomorphic agent library 107 can employ a trained and instantiated agent 108 to play with the user when the user first launches a new game.

[0025] Those skilled in the art will understand that the system 100 is operable to provide an anthropomorphic agent or artificial intelligence capable of learning a player's uniqueness, learning a player's communication style or tendencies, learning strategies employed and used by the player in various different games and scenarios, and learning the mechanics of game play for specific games and game genres, etc. Further, one or more agents generated by the system 100 can be stored as part of a cloud service that enables the system to maintain a "memory" of the user's past interactions, thereby enabling the system to generate an agent that functions as a consistent companion for the user across various games without the game having to be specially designed to support such an agent.

[0026] Figure 1B shows an overview of an example of a system 150 for generating and utilizing responses of anthropomorphic agents in a game system. As shown in system 150, two players, player 1 152 and player 2 154, are interacting with one or more agents 156. Although two players are shown, one of ordinary skill in the art will understand that any number of players can participate in a game session using system 150. A helper service 158 is provided to assist in anthropomorphizing the interaction of one or more agents 158 with an individual player or simultaneously with multiple players. As discussed in Figure 1A, one or more models 166 can be used to generate or modify a prompt 168. The prompt 168 is provided to the helper service 158, which applies several engines (e.g., agent persona engine 160, user goal or intent engine 162, and game lore or constraint engine 164) to modify the prompt and provide a more anthropomorphic interaction with an individual player or group of players.

[0027] For example, the Agent Persona Engine 160 can modify or adjust a prompt (or an action determined by the prompt) according to anthropomorphic information related to the agent. For example, a user can adopt an agent with a preferred personality. The Agent Persona Engine 160 can modify a prompt or a response generated by the prompt according to the personality of the agent. The User Intent or Goal Engine can modify a prompt (or an action determined by the prompt) based on the user's current goal or the intent behind the user's action or request. The user's goal or intent may change over time, may be based on a specified user goal, or may be determined based on the user's action. The Game Lore or Constraint Engine 164 can modify or adjust a prompt (or an action determined by the prompt) according to the characteristics of the game. For example, an agent may be "possessed" by a non-player character in the game (as discussed in more detail below). The Game Lore or Constraint Engine 164 can modify a prompt based on the personality or limitations of the NPC. The various engines of the helper service can be adopted individually or in combination when modifying or adjusting a prompt. Thereafter, the adjusted prompt is provided to one or more related agents 156 for execution.

[0028] Figure 2 shows an example of a method 200 for generating a humanoid agent. For example, method 200 may be employed by system 100. The flow starts with an operation 202 of instantiating a game session or receiving an indication that a game session is being instantiated. As described above, the game session may be hosted by the system executing method 200 or by a user device such as a game console. When the game is instantiated or an indication that a game session has been established is received, the flow proceeds to an operation 204 of instantiating an agent as part of the game session. In one example, the agent can be instantiated in response to receiving a request to add an agent to the game session. For example, a request to instantiate an agent within a multiplayer game or an agent that controls an NPC or AI companion within a single-player game may be received. Instantiating the agent may include identifying an agent related to the user playing the game from an agent library. As described above, aspects of the present disclosure enable the generation of agents playable across various games. Thus, the agent instantiated in operation 204 can be instantiated using various components stored as part of a humanoid agent library. For example, the agent can be instantiated using a machine learning model trained to perform game-specific mechanisms and actions, anthropomorphic characteristics learned over time through interactions with play, game data, or lore stored for a particular game or genre, and the agent is instantiated or trained based on a game genre similar to the game in which the agent is instantiated. As discussed earlier, the selected agent can be anthropomorphized to suit the user playing the game based on past interactions with the user in the same or a different game as the one started in operation 202.Alternatively, instead of dynamically instantiating an agent using various components stored within an agent library, a specific agent can be selected at operation 204. That is, the agent library may include specific “builds” for various types of agents that are designed by a user or derived from a specific gameplay with the user. These agents are saved and can be instantiated by the user in a future game session of the same game in which they were first created or in a different game. When an agent is instantiated at operation 204, the agent participates in the game session with the user.

[0029] In operation 206, the current game state is interpreted based on audio and visual data and / or via API access granted to the agent by the game. As described above, certain aspects of the present disclosure enable the generation of agents that can interact with a game without requiring API access or program access to the game. Thus, the instantiated agent interacts with the game in the same way as the user does, i.e., via audio and visual data related to the game. Alternatively, for example, if the agent relies on an NPC to interact with the game, API access to the game can be granted to the agent. In operation 206, various speech recognition, computer vision, object detection, OCR processes, etc. can be employed to process the game state in order to interpret communications received from the player (e.g., spoken commands, text-based commands) and the current display view (e.g., using computer vision) or an API for the game state. The current game state is used in operation 208 to generate agent actions. For example, agent actions can be executed based on spoken commands received from the user. Alternatively, agent commands can be generated based on the current view. For example, when an enemy appears on the screen, the enemy can be identified using computer vision and / or object detection, and in operation 208, the agent can generate a command to attack the enemy. Although not shown, operations 206 and 208 can be continuously executed while the game session is active.

[0030] The flow proceeds to operation 210 of receiving user feedback. The user feedback received can be explicit. For example, the user may issue a specific command to the agent to perform an action or to change the action currently being performed. Alternatively or in addition, the user feedback may be implicit. Implicit user feedback can be feedback data generated based on the user's interaction with the game. For example, the user may not explicitly provide a command to the agent and instead can adjust their actions or play style based on the current game state and / or in response to the actions performed by the agent. In an example, the user feedback can be continuously collected during a game session. The collected feedback can be related to the concurrent game state or agent actions.

[0031] When user feedback is collected, the flow proceeds to operation 212 of generating a prompt for one or more agents based on the user feedback. In the example, the generated prompt is an instruction to execute an agent action in response to the state of the game or a specific user interaction. The prompt can be generated using one or more machine learning models that receive user feedback, and / or actions performed by one or more agents, and / or existing prompts, and / or state data. The output of the machine learning model can be used to generate one or more prompts. In the example, the machine learning model can be trained using information related to the user so that the output from the machine learning model is anthropomorphized for the user. Alternatively or in addition, the machine learning model can be trained with respect to a specific game or application, with respect to a specific group of users (e.g., an e-sports team), etc. Multiple machine learning models can be employed in operation 212 to generate the prompt. In yet other examples, in addition to or instead of using a machine learning model in operation 212, other processes such as a rule-based process may be employed. Further, in operation 212, a new prompt may be generated or an existing prompt may be modified.

[0032] When one or more prompts are generated in operation 212, the flow proceeds to operation 214 where one or more prompts are stored for future use by one or more agents. For example, one or more prompts can be stored in an agent library. By storing the prompts generated in 212 using the agent library, the agents can utilize the prompts to interact with the user across various games, thereby providing an anthropomorphic agent that the user can play across various games.

[0033] FIG. 3 shows an exemplary system 300 for instantiating anthropomorphic agents that participate in gameplay with a user. For clarity of explanation, system 300 is described as including specific devices that perform the actions to be described in a specific order. However, without departing from the scope of the present disclosure, other devices may be included as part of system 300, or the actions to be described may be performed in a different order, and / or the requirements to be described may arise from or be sent to different devices, as will be understood by those skilled in the art. As shown in FIG. 3, system 300 may include a user device 302, a game platform 304, and an anthropomorphic agent service 306. Although not shown, in one aspect, user device 302, game platform 304, and anthropomorphic agent service 306 may be separate devices that communicate over a network. Alternatively, the components of user device 302, game platform 304, and anthropomorphic agent service 306 may be on the same device or on a network of devices. For example, game platform 304 and anthropomorphic agent service 306 may be part of the same cloud network.

[0034] The user device 302 can be a game console, a personal computer, a smartphone, a tablet, or any other type of device capable of running a game application. The game platform 304 can be one or more servers or a cloud network that supports game services. Exemplary services supported by the game platform 304 can include, for example, hosting online multiplayer games, distributing digital media, managing licenses and rights, managing a friends list, enabling communication between players, and the like. Examples of game platforms include, but are not limited to, XBOX LIVE, STEAM, PLAYSTATION NETWORK, BATTLE.NET, and the like. The anthropomorphic agent service 306 can be a server or a cloud network that can train and maintain a library of anthropomorphic agents for users that can be adopted across a wide variety of different games. Although illustrated as a separate entity or network, in some aspects the anthropomorphic agent service 306 may be part of the game platform 304 or share the same network as the game platform 304.

[0035] The user device 302 can execute a player game instance 308 for the user. The player game instance can be any multiplayer (or in some cases single-player) game. A user participating in the player game instance 308 can access contacts or a friend list hosted by the game instance or the game platform 304. For example, the user device 302 may include a game service interface / client 310 that enables the user to access their own contacts or friend list hosted by the game platform 304. In one example, the game service / client interface 308 can be a component of the operating system of the user device 302, such as the operating system of a game console. Alternatively or in addition, the game service / client interface 308 can be part of an application on the user device 308, such as a client-side application for the game platform 304. As an example, a user may want to invite a friend to participate in a game session related to the player game instance 308. To do so, the user can access their contacts or friend list through the player game instance 308 or via the game service interface / client 310 to check if any of their friends are online. In some examples, a user may play when none of their friends respond, when no one else wants to play the same game as the user, or when none of the user's friends have the character or class designation required to play with the user. Through integration with the anthropomorphic agent service 306, aspects of the present disclosure can also display one or more anthropomorphic bots associated with the user as part of the user's contacts or friend list. Thus, if the user cannot find a friend who responds or is willing to play, the user can invite one or more of their anthropomorphic agents to play with themselves.In one example, for instance, by selecting a particular agent from the user's friend list, the user can create one or more anthropomorphic agents to invite other human players to participate in the game session. For example, the user can have one or more predefined agents that can be directly selected from the friend list. Alternatively or in addition, in another example, as part of inviting an anthropomorphic agent, the user can select or provide additional parameters related to the agent's characteristics such as specific personality traits or abilities, specific strategies, specific playstyles, specific roles, etc. These parameters can be related to the invitation for the anthropomorphic agent to participate in the user's game session. The invitation to the agent can be provided to the game platform 304 via the game service interface / client 310.

[0036] As discussed herein, aspects of the present disclosure relate to enabling a user to invite a humanoid agent to join in a game play session without requiring individual games to explicitly support the creation of agents or the game play of agents. Thus, from the perspective of the game the user is playing, adding a humanoid agent cannot be distinguished from adding a new human player to the game. As a result, for a humanoid agent to participate in a game, a new instance of the game is required, just as a human player participating must have access to play the game (e.g., by owning a copy of the game, having a license for the game, having a subscription to the game, etc.). The game platform 304 may include an agent invitation interface 312 operable to receive a request to invite an agent from the user device 302. In response to receiving a request to add a humanoid agent from the user device 302, the game platform 304 can be determined by the agent invitation interface 312 that the invitation is for a humanoid agent (as opposed to another human player), for example, by identifying the agent within the invitation, additional parameters associated with the invitation, etc. When it is determined that the invitation is for a humanoid agent, the game platform 304 can determine whether the user of the humanoid agent has a license to play the game. For example, the licensing manager 314 can perform a search to determine whether the user has multiple licenses for the game associated with the request to invite the humanoid agent. If the user has multiple licenses for the game, or has an account that has received an additional license for the game, the humanoid agent can use one of those additional licenses to participate in the game. Alternatively, the game platform 304 or the game itself may be able to provide an agent license or enable the user to purchase an agent license.An agent license may be limited to enabling an agent (rather than another human player) to execute an instance of the game in order to play with the user. Since the license is limited to the agent, the agent license may be provided to the user at a different fee than a standard game license (e.g., it can be lower than a standard game license, it can be higher than a standard game license, a fixed fee that increases or decreases when the user purchases an additional agent license, an initial game license, etc.). If the license manager 314 cannot find an additional user or agent license, the game platform 304 can transmit a request to the user device 302, for example, via the agent invitation interface 312, and cause the user device 302 to prompt the user to purchase an additional game license or agent license. When the purchase of the additional game or user license is successful, the additional game or user license can be added to the user's account, for example, by the license manager 304. If the user has an additional game license or agent license, the license manager 314 can obtain any rights and / or license information necessary to instantiate an additional game session for the agent, and the game platform 304 can transmit the rights and / or license information to the user device 302, thereby enabling the user device 302 to instantiate a new instance of the game for the agent.

[0037] When an appropriate license is verified by the game platform or the game itself, the game platform may generate a request for an anthropomorphic agent. For example, game platform 304 may aggregate anthropomorphic agent information (e.g., an anthropomorphic agent identifier identifying a specific anthropomorphic agent, specific personality traits or abilities, specific strategies, specific playstyles, specific roles, etc.) from the agent invitation request from the request. This information can be included as part of a request to create and / or instantiate an anthropomorphic agent within a newly created game session. Game platform 304 can also access additional information regarding the game in which the user or agent participates from account / game data store 316. Account / game data store 316 can store information regarding the user, such as the user's ID, gamer tag, the user's player character information (e.g., class, role, status, characteristics, etc.). This user information can also be included in a request to create and / or instantiate an anthropomorphic agent. In a further example, the game platform may also access information specific to the game, such as the game server or IP address associated with the game play session, account information related to the character of the anthropomorphic agent within the game (e.g., the level, ability, role, etc. of the anthropomorphic agent character). The information within the agent invitation request from user device 302, the user and / or game information from account / game data store 316, and / or the rights information of anthropomorphic agent 318 can be aggregated by agent service interface 318 and packaged into a request to create and / or instantiate an anthropomorphic agent. Agent service interface 318 can then send the request to create and / or instantiate an anthropomorphic agent to anthropomorphic agent interface 306.

[0038] The anthropomorphic agent interface 306 is operable to receive requests from the game platform 304 via the agent request interface 320 to create and / or instantiate an anthropomorphic agent. The agent request interface 320 is operable to extract request parameters (e.g., information related to the requested characteristics of the anthropomorphic agent, user information, and / or game information), and provide the request parameters to the anthropomorphic agent selector 322. The anthropomorphic agent selector 322 analyzes the request parameters and identifies the requested agent characteristics based on the request parameters. For example, the anthropomorphic selector can identify requests for specific characteristics from the request parameters. Additionally, the anthropomorphic selector 306 may infer characteristics based on the request parameters. For example, information about the game, such as the type of game or the genre of the game, may be used to identify agent characteristics related to the game or NPC. For example, if the game is a role-playing game, the anthropomorphic selector 306 can identify agent characteristics related to or useful for the role-playing game. Similarly, if the game is a first-person shooting game, the anthropomorphic selector 306 can identify agent characteristics related to the first-person shooting game. Similarly, the in-game characteristics of the anthropomorphic agent character can be used to infer related agent characteristics. For example, if the in-game character of the anthropomorphic agent is a healer, the anthropomorphic selector can identify agent characteristics related to the healer class. Once the related agent characteristics are identified, agent data related to the related agent characteristics is retrieved from the anthropomorphic agent data store 324. In an example, the anthropomorphic agent data store 3247 can store any type of anthropomorphic information (e.g., prompts, machine learning models, personality traits of the agent, etc.) generated in the past for the agent. The retrieved anthropomorphic information is aggregated from the anthropomorphic agent data store 324 and can be disclosed (e.g., transmitted or provided access) to the user device 302 via, for example, the agent request interface 320.

[0039] Upon receiving license and / or right information from the game platform 304, the user device 302 can execute a virtual machine on the client device. The virtual machine can be used to execute a second game instance for the agent character while simultaneously executing the player game instance 308 on the user device 302. When establishing a virtual machine that executes the anthropomorphic agent game instance 328, the user device can utilize the anthropomorphic agent information published by the anthropomorphic agent service 306 via the agent request interface 320 to create an instance of the anthropomorphic agent within the game session executed within the virtual machine 328 based on the received anthropomorphic agent data. Therefore, the client device 302 may not need to execute two different game instances, one for the user's player character and one for the anthropomorphic agent character. From a game perspective, it would be as if two different game instances were being operated by two different players. That is, the anthropomorphic agent can interact with the game without the game needing to provide support for the agent to interact (e.g., without creating a specific API to enable the anthropomorphic agent to interact with the game). Therefore, when the user's friend is not online or cannot play the game with the user, the multiplayer functionality of the game can be utilized without modification to allow the user to invite their anthropomorphic agent to the game session. However, since the game does not need to expose an API to enable agent interaction, the anthropomorphic agent can interact with the game and participate in the game as a normal human player. That is, the anthropomorphic agent can receive game data via vision, voice, and text available to human users, process the received data using one or more machine learning models to determine the current game state, and determine appropriate actions to execute based on the current game state. In some examples, this process can be executed on the user device 302.However, in many cases, the user device 302 may not have the computational resources necessary to execute two different game sessions and one or more machine learning models to simultaneously control the gameplay of the anthropomorphic agent, and even if it were possible, it could not be executed without causing delays that would adversely affect the user's gameplay experience. Therefore, the anthropomorphic agent service 306 can control the gameplay of the anthropomorphic agent via the anthropomorphic agent game interface 326.

[0040] In certain aspects, the anthropomorphic agent service 306 can employ an anthropomorphic agent game interface 326 operable to receive current game data (e.g., video data, audio data, text data, communication between players, tactile information, etc.) generated by an agent game instance running within the virtual machine 328. The current game data received by the game interface 326 can be processed using one or more machine learning models and / or other components of the anthropomorphic agent library discussed in FIG. 1A to interpret the current game state, determine actions of the anthropomorphic agent based on the current game state, and transmit control instructions for controlling the gameplay of the instantiated anthropomorphic agent.

[0041] Although certain functions have been described as being performed by specific devices or components of system 300, one of ordinary skill in the art will understand that other devices or components that are part of system 300 or other devices not shown in FIG. 3 can perform the actions and functions described without departing from the scope of the present disclosure. For example, although system 300 is depicted as having both a player game instance 308 and a virtual machine game instance 328 being executed on user device 302, in other embodiments these two game instances may be executed on other devices. For example, the game instances may be executed on game platform 304 or on a game server (not shown in FIG. 3) that hosts a game session.

[0042] Figure 4 shows an exemplary method 400 for creating an anthropomorphic agent game session and instantiating an anthropomorphic agent within the agent game session. In one example, method 400 may be executed by a user device that executes a game session for the user's player character. In an alternative example, method 400 may be executed using other devices, components, or cloud services described herein. The flow begins with operation 402 of receiving a request to invite an anthropomorphic agent to participate in a game play with the user. The request may be received via a contact or friend invitation interface. For example, the user can open their friend list and select one or more anthropomorphic agents to invite to the game. As discussed above, this request can be for a specific anthropomorphic agent and / or can include characteristics of the anthropomorphic agent. Upon receiving the request, the flow proceeds to operation 404, where a request to have the anthropomorphic agent participate in the game session is generated and transmitted to the game service. In operation 404, details regarding the anthropomorphic agent (e.g., identifier of a specific anthropomorphic agent, characteristics of the anthropomorphic agent, etc.), information regarding the user (e.g., user identifier that can be used to associate the user with their anthropomorphic agent or anthropomorphic agent data, character information of the user, etc.), and / or details regarding the game (e.g., game identifier, world or server identifier indicating the server on which the user is currently playing, etc.) can be aggregated and transmitted to a game service and / or an anthropomorphic agent game service that manages the user's contacts or friend list, the user's rights, the user's multiplayer capabilities, etc.

[0043] In response to transmitting the request, the flow proceeds to operation 406 where the apparatus executing method 406 receives rights for the anthropomorphic agent game session. Although not shown, as discussed above, a prompt may be generated to request the user to obtain an additional game license or an additional agent license as described above before receiving the rights in order to execute an additional game session. Upon receiving the rights, the apparatus executing method 400 can execute an additional instance of the game and have the data necessary to connect the game to an online game service. The flow then proceeds to operation 408 where the apparatus executing method 400 creates a virtual environment for executing the additional game session. The virtual environment (e.g., virtual machine, container, etc.) enables the apparatus executing method 400 to simultaneously execute two different game sessions for the same game (e.g., a game session for the user and a game session for the anthropomorphic agent).

[0044] In addition, as discussed above, the apparatus executing method 400 can receive information regarding the anthropomorphic agent, and this information enables the apparatus to instantiate the anthropomorphic agent within the newly created game session within the virtual environment. The anthropomorphic agent information used to instantiate the anthropomorphic agent can be received from an anthropomorphic agent service, a game platform, or a combination of the two. In one example, the anthropomorphic agent information can identify a character within the game of the anthropomorphic agent. Thus, the character within the game of the anthropomorphic agent can be selected in operation 410 to instantiate the anthropomorphic agent within the newly created game instance. Alternatively or in addition, the anthropomorphic agent data can include information regarding the characteristics of the agent, game data, components that control the gameplay of the anthropomorphic agent (e.g., one or more machine learning models), etc. This information may be used in addition to or instead of the anthropomorphic agent's in-game character identifier for instantiating the anthropomorphic agent within the game.

[0045] As described above, since the API access may not be provided by the game to enable the anthropomorphic agent to directly interact with the game, the anthropomorphic agent can interact with the game in the same way as a human player (e.g., by interpreting the current game state from the visual and audio parts of the game and generating actions in response). However, the apparatus executing method 400 may not have the computing resources to support two different game sessions and execute the machine learning model required to control the game play of the anthropomorphic agent. Therefore, in operation 412, for example, a connection with the anthropomorphic agent service can be established. This connection can transmit the current game state (e.g., visual, audio, tactile, player communication, etc.) to the anthropomorphic agent service and receive, in response, a control signal for controlling the game play of the anthropomorphic agent within the game instance executed within the virtual environment. This connection persists during the game play of the anthropomorphic agent, thereby enabling the anthropomorphic agent to continue playing with the user during the game session.

[0046] Figure 5 shows an exemplary method 500 for determining whether a session of a humanoid agent game can be established. In one example, method 500 can be executed by a game platform. In an alternative example, method 500 can be executed using other devices, components, or cloud services, or combinations thereof, as described herein. The flow begins with operation 502 of receiving a request to invite a humanoid agent to a game. For example, this request can be received from a user device that is running a game session for the user. In various aspects, details regarding the humanoid agent (e.g., an identifier of a particular humanoid agent, characteristics of the humanoid agent, etc.), information regarding the user (e.g., a user identifier that can be used to associate the user with their humanoid agent or humanoid agent data, character information of the user, etc.), and / or details regarding the game (e.g., a game identifier, a world or server identifier indicating the server on which the user is currently playing, etc.) can be included as parameters associated with the received request.

[0047] The flow proceeds to operation 504 of checking a license repository to determine whether the user requesting the agent has the licenses required to establish a game session for that agent. In various aspects, the license repository can be queried using information regarding the user and game associated with the request to determine whether the user has an additional license for the game or an unused agent license for the game or game platform. Based on the result of the query, a determination is made at operation 510 as to whether the user has the required license. If the user does not have the correct license, the flow branches to operation 508 with a no, where a command is transmitted to the requesting device to prompt the user to obtain (e.g., purchase an additional game license or agent license). Thereafter, the flow returns to operation 504 (or 502) to continue the process until the required license is found in the repository.

[0048] If the user has the required game license or agent license, the flow branches from the Yes operation 506 to operation 510. In operation 510, the rights required to establish an additional game session for the anthropomorphic agent are transmitted to the requesting device. As discussed above, the aforementioned rights can be used by the requesting device to establish a new game instance and connect to the game's services. The flow proceeds to operation 512, where it can collect user and game information from one or more data stores managed by the game platform or the game itself. The information collected in operation 512 may relate to aspects of the game necessary for the agent to anthropomorphize the agent for the game the user is playing and / or to connect the agent to the correct server or world to enable the anthropomorphic agent to play with the user. In operation 506, the collected information is aggregated and transmitted to the anthropomorphic agent service and / or the requesting user device.

[0049] FIG. 6A shows an exemplary method 600 for instantiating and controlling a humanoid agent within a humanoid agent game session. In one example, method 500 may be executed by a game platform. In an alternative example, method 500 may be executed using other devices, components, or cloud services, or combinations thereof, as described herein. The flow begins with operation 602 of receiving a request to instantiate a humanoid agent within a game session. The request may include information about an existing humanoid agent, such as an identifier that identifies a particular agent. Alternatively or in addition, the request may include the requested characteristics of the agent, characteristics regarding the user and / or the user's player character within the game, and / or information about the game in which the humanoid agent is to be instantiated. The flow proceeds to operation 604 of analyzing the request parameters to identify a particular humanoid agent and / or the requested or relevant characteristics regarding the humanoid agent. For example, the request parameters can be analyzed for particular characteristics to include in the humanoid agent. In addition, method 600 may infer characteristics based on the request parameters. For example, information about the game, such as the type or genre of the game, may be used to identify agent characteristics relevant to the game. For example, if the game is a role-playing game, the humanoid selector 306 can identify agent characteristics relevant to or useful for role-playing games. Similarly, if the game is a first-person shooting, the humanoid agent selector 322 can identify agent characteristics related to first-person shooting games. Based on the analysis, the specific humanoid agent and / or the characteristics used to create the humanoid agent for the requested game session can be collected from the humanoid agent data store related to the user playing the game. In operation 606, aggregated information regarding the humanoid agent is transmitted to the device hosting the game session of the humanoid agent, along with instructions to instantiate the humanoid agent within the game session.For example, an instruction such as an instruction for creating a new character controlled by an anthropomorphic agent that can be used to instantiate a character in a game related to a past game session of the anthropomorphic agent character. In one example, anthropomorphic agent data and instructions for instantiating an anthropomorphic agent within a game session can be sent to a device executing the user's game session and / or a device executing the game session of the anthropomorphic agent.

[0050] As discussed, from a game perspective, two different instances of games are being run by two different players currently. That is, the anthropomorphic agent can interact with the game without the game needing to provide support for the agent to interact (e.g., without creating a specific API to enable the anthropomorphic agent to interact with the game). Therefore, if the user's friend is not online or cannot play the game with the user, the multiplayer function of the game can be utilized without modification to allow the user to invite their anthropomorphic agent to the game session. However, since the game does not need to be made public, and there is an API to enable agent interaction, the anthropomorphic agent can interact with and engage with the game as a normal human player. That is, the anthropomorphic agent is operable to receive game data via vision, voice, and text available to human users, process the received data using one or more machine learning models to determine the current game state, and determine appropriate actions to execute based on the current game state. Thus, in operation 608, a communication session is established with a device that hosts the game session of the anthropomorphic agent, such as the user device of FIG. 3 or another device. This communication session is used to receive game state information in the form of audio data, visual data, tactile data, text data, etc. Further, the communication session established in operation 608 can be used by the device executing method 600 to transmit control instructions for controlling the gameplay of the anthropomorphic agent.

[0051] Once a communication session is established, the flow proceeds to operation 610 where the device executing method 600 receives game data (e.g., visual, audio, tactile data) generated by the game session of the anthropomorphic agent. The game data received in operation 610 can be the same as the game data available to a human player. That is, the game data need not include API access to game data not available to a human player. In operation 612, the received game data is analyzed using one or more machine learning models. For example, the received game data can be provided to one or more basic machine learning models, object recognition models, speech recognition models, natural language understanding models, etc. to process the current game state. In operation 614, the output of one or more machine learning models is used, either alone or in further processing using other components described as part of the anthropomorphic agent library 107 of FIG. 1, to generate instructions for controlling the interaction of the anthropomorphic agent during gameplay according to the current game state. The foregoing actions can include, for example, where to move the character of the anthropomorphic agent, what abilities or actions the character of the anthropomorphic agent should perform, accessing the inventory of the character of the anthropomorphic agent, etc. That is, any action executable by a player within the game is determined in operation 614 and transmitted to the device hosting the game session of the anthropomorphic agent, thereby enabling the instance of the anthropomorphic agent to execute the in-game action. By doing so, the anthropomorphic agent can interact and participate within the game without requiring the game to be specially developed to support the anthropomorphic agent. Therefore, the user can adopt in the future anthropomorphic agents developed over time by continuously playing the game to play against one or more anthropomorphic agents within the same or different games.

[0052] The flow proceeds to operation 616 that determines whether the game is still in session. If it is in session, the flow branches to operation 610, where the game data is continuously received and processed by the apparatus executing method 600 to enable continuous gameplay between the anthropomorphic agent and the user. However, if the game has ended, the flow branches to the yes operation 618, where the communication session ends and the control of the anthropomorphic agent stops.

[0053] FIG. 6B shows an example of a method 650 that enables an anthropomorphic agent to interact with a user during gameplay using computer vision. The flow starts from operation 652 that receives an interaction from the user. The interaction can be a speech or text-based interaction. For example, the user can ask "Does this object look like a house?" via a voice interface or a chat interface. Alternatively, the interaction can be received by a user action rather than a user communication. Upon receiving the interaction, the flow proceeds to operation 654, where the interaction is analyzed to determine the user's intent related to the user's request and / or action. For example, if the interaction is a user communication, the communication can be processed to determine the request made by the user, and a speech recognition and / or natural language understanding model can be used to identify the intent related to the request. If the interaction is a user action, other techniques such as computer vision techniques and event logging techniques can be employed to determine whether the intent or action behind the action implies the user's request.

[0054] Having determined the request and / or intent, the bot may visually process the game environment at operation 656 to determine a response to the request. That is, rather than accessing game data, the bot may use computer vision techniques to visually inspect the game's surroundings and determine the state of the game as a human user would. For example, if the user's request is "does this look like a house", the bot may use computer vision and object detection techniques to analyze the objects around it and identify those that look like houses, rather than accessing game or state data, for example, to determine if any of the objects in the vicinity have been tagged or identified as houses. Based on the analysis, flow proceeds to operation 658 where the bot generates a response to the request based on information determined from visually analyzing the game environment.

[0055] Although the particular examples described herein relate to utilizing agents within a gaming environment, those skilled in the art will appreciate that the techniques described above can be applied to create and use agents within other types of environments, such as corporate environments, etc. For example, anthropomorphic agents can be created to assist users in performing tasks within a corporate environment, or for use with any other type of application.

[0056] FIG. 7 is a block diagram showing the physical components (e.g., hardware) of a computing device 700 in which aspects of the present disclosure may be practiced. The components of the computing device described below may be suitable for the computing device above. In a basic configuration, the computing device 700 may include at least one processing unit 702 and a system memory 704. Depending on the configuration and type of the computing device, the system memory 704 may include, but is not limited to, a volatile storage area (e.g., random access memory), a non-volatile storage area (e.g., read-only memory), flash memory, or any combination of such memories. The system memory 704 can include an operating system 705 and one or more program tools 706 suitable for executing the various aspects disclosed herein. The operating system 705 may be suitable for controlling the operation of the computing device 700, for example. Further, aspects of the present disclosure may be practiced with a graphics library, other operating systems, or any other application program, and are not limited to a particular application or system. This basic configuration is shown in FIG. 7 by the components within the dashed line 708. The computing device 700 can have additional features or functionality. For example, the computing device 700 may also include additional (removable and / or non-removable) data storage devices such as, for example, magnetic disks, optical disks, or tapes. Such additional storage areas are shown in FIG. 7 by the removable storage device 709 and the non-removable storage device 710.

[0057] As described above, several program tools and data files can be stored within system memory 704. While being executed on at least one processing unit 702, program tool 706 (e.g., application 720) can execute a process including, but not limited to, aspects described herein. Application 720 includes a humanoid agent generator 730, a machine learning model 732, a game session 734, a humanoid agent controller 736, and instructions for executing various processes disclosed herein. Other program tools that can be used in accordance with aspects of the present disclosure can include, for example, an email and contact application, a document processing application, a spreadsheet application, a database application, a slide presentation application, a drawing or computer-aided application program, and the like.

[0058] Aspects of the present disclosure can be practiced within an electrical circuit including discrete electronic elements, a packaged or integrated electronic chip including logic gates, within a circuit utilizing a microprocessor, or on a single chip including an electronic element or microprocessor. For example, aspects of the present disclosure can be practiced by a system-on-chip (SOC), in which each or many of the components shown in FIG. 7 can be integrated on a single integrated circuit. Such an SOC device can include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned in") on a chip substrate as a single integrated circuit. When operating by an SOC, the functions described herein with respect to the ability of a client to switch protocols can operate by application-specific logic integrated with other components of the computing device 700 on a single integrated circuit (chip). Aspects of the present disclosure can also be practiced using other technologies capable of performing logical operations such as AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum technologies. Additionally, aspects of the present disclosure can be practiced within a general-purpose computer or within any other circuit or system.

[0059] The computing device 700 can also include one or more input devices 712 such as a keyboard, a mouse, a pen, a voice or voice input device, a touch or swipe input device. It can also include an output device 714 such as a display, a speaker, a printer, etc. The foregoing are examples, and other devices can also be used. The computing device 700 can include one or more communication connections 716 that enable communication with other computing devices 750. Examples of communication connections 716 include, but are not limited to, radio frequency (RF) transmitters, receivers, and / or transceiver circuits, universal serial bus (USB), parallel and / or serial ports.

[0060] As used herein, the term computer-readable medium may include computer storage media. Computer storage media can include volatile and nonvolatile removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, or program tools. System memory 704, removable storage device 709, and non-removable storage device 710 are all examples of computer storage media (e.g., memory storage areas). Computer storage media can include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical storage areas, magnetic cassettes, magnetic tape, magnetic disk storage areas or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computing device 700. Such any computer storage media can be part of computing device 700. Computer storage media does not include carrier waves or other propagated data signals or modulated data signals.

[0061] Communication media may be embodied by computer-readable instructions, data structures, program tools, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" may refer to a signal in which one or more characteristics are set or changed in such a manner as to encode information in the signal. By way of example and not limitation, communication media can include wired media such as a wired network or direct wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0062] Figures 8A and 8B illustrate a computing device or mobile computing device 800 in which aspects of the present disclosure may be practiced, such as a cellular phone, smartphone, wearable computer (such as a smartwatch), tablet computer, laptop computer, and the like. In some aspects, a client utilized by a user (such as client device 102 shown within system 100 of FIG. 1) may be a mobile computing device. Referring to FIG. 8A, one aspect of a mobile computing device 800 for implementing aspects is shown. In a basic configuration, mobile computing device 800 is a handheld computer having agent input and output elements. Mobile computing device 800 typically includes a display 805 and one or more input buttons 810 that enable a user to input information into mobile computing device 800. The display 805 of mobile computing device 800 may also function as an input device (such as a touch screen display). When included as an optional input element, side input element 815 enables additional user input. Side input element 815 may be a rotary switch, button, or any other type of manual input element. In alternative aspects, mobile computing device 800 may incorporate more or fewer input elements. For example, in some aspects, display 805 may not be a touch screen. In yet another alternative aspect, mobile computing device 800 is a cellular phone system such as a cell phone. Mobile computing device 800 may also include an optional keypad 835. Optional keypad 835 may be a physical keypad or may be a "soft" keypad generated on a touch screen display. In various aspects, output elements include a display 805 for displaying a graphical user interface (GUI), visual indicator 820 (such as a light emitting diode), and / or an audio transducer 825 (such as a speaker). In some aspects, mobile computing device 800 incorporates a vibration transducer for providing haptic feedback to the user.In yet another aspect, the mobile computing device 800 incorporates input ports and / or output ports such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., an HDMI port) for transmitting signals to or receiving signals from an external device.

[0063] FIG. 8B is a block diagram showing an architecture of one aspect of a computing device, a server (e.g., the application server 104, the incident data server 106, and the incident correlator 110 shown in FIG. 1), a mobile computing device, etc. That is, the mobile computing device 800 can incorporate a system 802 (e.g., a system architecture) for implementing some aspects. The system 802 can be implemented as a “smartphone” capable of executing one or more applications (e.g., a browser, email, a calendar, contact management, a messaging client, a game, and a media client / player). In some aspects, the system 802 is integrated as a computing device such as a personal digital assistant (PDA) and a wireless phone.

[0064] One or more application programs 866 may be loaded into the memory 862 and executed on or in relation to the operating system 864. Examples of application programs include a telephone dialer program, an email program, a personal information management (PIM) program, a document processing program, a spreadsheet program, an Internet browser program, a messaging program, and the like. The system 802 also includes a non-volatile memory area 868 within the memory 862. The non-volatile memory area 868 can be used to store persistent information that should not be lost even if the power of the system 802 is turned off. The application program 866 can use and store information within the non-volatile memory area 868 such as emails or other messages used by an email application. A synchronization application (not shown) is also on the system 802 and is programmed to interact with a corresponding synchronization application on a host computer to keep the information stored in the non-volatile memory area 868 synchronized with the corresponding information stored on the host computer. As should be understood, other applications may be loaded into the memory 862 and executed on the mobile computing device 800 described herein.

[0065] The system 802 has a power source 870 that can be implemented as one or more batteries. The power source 870 may further include an external power source such as an AC adapter or a powered docking station that supplements or charges the battery.

[0066] The system 802 can also include a wireless interface layer 872 that performs the function of transmitting and receiving radio frequency communications. The wireless interface layer 872 facilitates a wireless connection between the system 802 and the "outside world" via a communication carrier or service provider. Transmissions to and from the wireless interface layer 872 are performed under the control of the operating system 864. In other words, communications received by the wireless interface layer 872 can be spread by the operating system 864 to the application program 866, and vice versa.

[0067] The visual indicator 820 (e.g., an LED) can be used to provide visual notification, and / or the audio interface 874 can be used to produce an audible notification by the audio converter 825. In the illustrated configuration, the visual indicator 820 is a light-emitting diode (LED), and the audio converter 825 is a speaker. These devices can be directly coupled to the power supply 870 to remain on for a period directed by the notification mechanism at startup, even if the processor 860 and other components may shut down to conserve battery power. The LED can be programmed to remain on indefinitely to indicate the powered-on state of the device until the user takes an action. The audio interface 874 is used to provide an audible signal to the user and to receive an audible signal from the user. For example, in addition to being coupled to the audio converter 825, the audio interface 874 may also be coupled to a microphone to receive an audible input to facilitate a telephone conversation. According to aspects of the present disclosure, the microphone may also function as an audio sensor to facilitate control of notifications as described below. The system 802 may further include a video interface 876 that enables the operation of a device connected to the peripheral device port 830 to record still images, video streams, etc.

[0068] The mobile computing device 800 implementing the system 802 can have additional features or functions. For example, the mobile computing device 800 can also include additional (removable and / or non-removable) data storage devices such as magnetic disks, optical disks, or tapes. Such additional storage areas are represented by the non-volatile storage area 868 in FIG. 8B.

[0069] Data / information generated or captured by the mobile computing device 800 and stored by the system 802 can be stored locally on the mobile computing device 800 as described above, or the data can be stored on any number of storage media, which can be accessed by the device via the wireless interface layer 872 or via a wired connection between the mobile computing device 800 and another computing device associated with the mobile computing device 800, such as a server computer within a distributed computing network such as the Internet. As should be understood, such data / information can be accessed by the mobile computing device 800 via the wireless interface layer 872 or via the distributed computing network. Similarly, such data / information can be easily transferred between computing devices for storage and use according to known data / information transfer and storage means, including email and collaborative data / information sharing systems.

[0070] In some examples, a system for controlling the gameplay of an anthropomorphic agent is disclosed. The system, when executed by at least one processor, receives from a user device a request to instantiate an anthropomorphic agent within an anthropomorphic agent gameplay session, where the anthropomorphic agent uses anthropomorphic agent data. The anthropomorphic agent data is for instantiating an agent within a gameplay session with a user for a first game, and the instantiated agent is operable to interact with a user who plays the first game based on one or more machine learning models trained to play the game. The operations include: instantiating, receiving user interaction during gameplay by the agent, generating an agent response to the user interaction by one or more machine learning models, instructing the agent to execute the agent response, receiving feedback from the user for the agent response, and generating agent anthropomorphic data based on the agent response and the user feedback. The operations also include: determining anthropomorphic agent data for the instantiated agent based on the request, sending the anthropomorphic agent data to the user device, establishing a communication session with the user device, receiving game data from the user device, analyzing the game data using one or more machine learning models to determine one or more actions to be executed by the anthropomorphic agent, and sending the one or more actions to the user device. The system includes a memory encoding computer-executable instructions for causing the at least one processor to perform the operations.

[0071] In some examples, the one or more machine learning models include a base model, a language model, a computer vision model, or a speech model.

[0072] In some examples, the video game data includes one or more of visual data, audio data, tactile feedback data, text data, or data published by an API.

[0073] In some examples, analyzing game data includes providing at least a portion of the video game data to a machine learning model trained to analyze a portion of game data, and determining a current game state based on the output of the machine learning model.

[0074] In some examples, one or more actions are determined based on the current game state.

[0075] In some examples, a request includes parameter data related to at least one of an identifier of a specific anthropomorphic agent, data related to one or more desired characteristics of the anthropomorphic agent, game data, or user data.

[0076] In some examples, the game data further includes at least one of a game identifier, information about the genre of the game, or the server or world in which the user character is playing.

[0077] In some examples, the user data includes at least one of data detailing the characteristics of the user's character or data detailing the user's preferred play strategy.

[0078] In some examples, a method is provided for instantiating an anthropomorphic agent within an instance of a game, the method comprising: executing a first instance of a game played by a user on a user device; receiving a request for the anthropomorphic agent to participate in the game; sending the request to a game service; in response to receiving the sending of the request, receiving rights for the anthropomorphic agent game instance; executing a second instance of the game for the anthropomorphic agent on the user device; receiving anthropomorphic agent data from an anthropomorphic agent service; and instantiating the anthropomorphic agent within the second instance of the game.

[0079] In some examples, the request is received by selecting an anthropomorphic agent from a friends list associated with the user.

[0080] In some examples, the selection is for a specific anthropomorphic agent.

[0081] In some examples, receiving the selection further comprises receiving one or more characteristics for the anthropomorphic agent.

[0082] In some examples, the rights for the anthropomorphic agent game instance are based on an agent license that is different from the game license for the game.

[0083] In some examples, the method further comprises establishing a communication session with the anthropomorphic agent service and sending game data to the anthropomorphic agent service via the communication session, the game data including at least one of visual data, audio data, or tactile data, and in response to sending the game data, receiving one or more commands for controlling the game play of the anthropomorphic agent.

[0084] In some examples, the second instance of the game is executed within a virtual environment on the user device.

[0085] In some examples, a method for determining whether a user has a license to instantiate an anthropomorphic agent, the method comprising receiving, from a user device, a request to invite an anthropomorphic agent to play a game with the user; determining whether the user has a license for a second instance of the game; if the user has a license for a second instance of the game, transmitting rights information to the user device, the rights information being operable to enable the user device to instantiate a second instance of the game; and transmitting a second request to an anthropomorphic agent service to cause instantiation of the anthropomorphic agent in the second instance of the game.

[0086] In some examples, determining whether the user has a license for a second instance includes querying a license repository to determine whether the user has a second license for the game.

[0087] In some examples, determining whether the user has a license for a second instance includes querying a license repository to determine whether the user has an agent license for the game.

[0088] In some examples, the agent license enables the user to execute a second instance of the game for use by the agent and does not enable the user to execute a second instance of the game for use by another human player.

[0089] In some examples, the agent license is obtained separately from a general license for the game.

[0090] The description and explanatory drawings of one or more aspects shown in this application are not intended to limit or restrict the scope of the present disclosure described in the claims in any way. The disclosure described in the claims should not be construed as being limited to, for example, any of the aspects or details provided in this application. Whether shown and described in combination or separately, various features (structural and methodological of the agent) are selectively included or omitted to create aspects having a particular set of features. Since the description and explanatory drawings of this application are provided, those skilled in the art can devise modifications, corrections, and alternative aspects that are within the spirit of the broader aspects of the general inventive concept embodied in this application without departing from the broader scope of the disclosure described in the claims.

Claims

1. At least one processor, When executed by at least one of the aforementioned processors, The user device running a first gameplay instance of the game receives a request to instantiate an anthropomorphic agent within an anthropomorphic agent gameplay session of a second gameplay instance of the game, wherein the anthropomorphic agent uses anthropomorphic agent data, and the anthropomorphic agent data is Instantiating an agent in a gameplay session with a user for the second gameplay instance, wherein the instantiated agent is operable to interact with the user playing the game through the first gameplay instance, based on one or more machine learning models trained to play the game. The agent receives user interactions during gameplay. The agent response to the user interaction is generated by the one or more machine learning models. Instructing the agent to execute the agent response using the second gameplay instance, Receiving feedback from the user regarding the agent response, and To generate agent personification data based on the agent response and user feedback. To be generated based on an action that includes receiving, To determine anthropomorphic agent data for the agent to be instantiated based on the above request, The anthropomorphic agent data is transmitted to the user device, To establish a communication session with the user device, Receiving game data from the user device corresponding to the first gameplay instance, Using one or more machine learning models, analyze the game data to determine one or more actions to be performed by the anthropomorphic agent, and Transmitting one or more of the aforementioned actions to the user device. A memory that encodes a computer executable instruction that causes the at least one processor to perform an operation including the above, A system that includes this.

2. The aforementioned one or more machine learning models Basic model, Language model, Computer vision models, or Speech model The system according to claim 1, including the following:

3. The aforementioned video game data Visual data, Audio data, Haptic feedback data, Text data, or Data exposed by the API The system according to claim 2, comprising one or more of the above.

4. Analyzing the aforementioned game data To provide at least a portion of the video game data to a machine learning model trained to analyze a portion of the game data, and The current game state is determined based on the output of the aforementioned machine learning model. The system according to claim 3, including the above.

5. The system according to claim 4, wherein one or more of the aforementioned actions are determined based on the current game state.

6. The above requirement is Identifier for a specific anthropomorphic agent, Data relating to one or more desired characteristics of the aforementioned anthropomorphic agent, Game data, or User data The system according to claim 1, comprising parameter data relating to at least one of the following.

7. Game data, Game identifier, Information about game genres, or The server or world in which the user character is playing The system according to claim 6, further comprising at least one of the following.

8. User data, Data detailing the characteristics of the user's character, or Data detailing the user's preferred play strategy. The system according to claim 6, comprising at least one of the following.

Images