Artificial Intelligence (AI) Model Training Using Cloud Gaming Network
By collecting game process data of multiple players on the server network, combining deep learning engines and success criteria, training AI models, the problems of long training time of AI model and difficulty in data collection are solved, and the understanding and functionality of game complexity are achieved response.
Patent Information
- Application Number
- CN202210481340.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-11-05
- Filing Date
- 2019-08-30
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2039-08-30
AI Technical Summary
The prior art is difficult to efficiently train artificial intelligence models, especially in generating and collecting large amounts of object image data, which leads to long and time-consuming training of AI models and difficult to adapt to object recognition at different angles and views.
By collecting and training AI models on the server network, using deep learning engines and success guidelines, combining data from multiple players during gameplay, training AI models for game applications, providing game-related functionality, such as recommended and weakness discovery.
It significantly reduces the time for training data collection, improves AI models' understanding of game complexity and analytical capabilities of gameplay, can provide appropriate responses based on predefined goals, and continuously improves training data.
Smart Images

Figure CN114949861B_ABST
Abstract
Description
[0001] JV Beltran, D. Butler, K. Kregenbrink
[0002] This application is a divisional application of the invention patent application with the application date of August 30, 2019, application number 201980086869.3, and invention name “Artificial Intelligence (AI) Model Training Using Cloud Gaming Network”. Technical Field
[0003] The present disclosure relates to artificial intelligence (AI) model training, and more particularly, to using a network of server consoles of a gaming cloud system to collect training data associated with remote players, the training data being used for AI model training. Background Art
[0004] Artificial intelligence algorithms are designed to learn from data. AI models can be built based on the data they learn from and are flexible enough to perform a variety of functions depending on the inputs provided to the AI model.
[0005] However, providing data to train AI models is complex. In a less simple example, a large number of object images are used to train an AI model configured to recognize objects. For example, a large number of object images are used to train the AI model. Generating and collecting those images is difficult and very time-consuming. Basically, the AI model is trained to recognize every type of object that exists. Imagine collecting multiple images of the object taken from different angles for each object. In this way, when a new image of the object is presented, the AI model can extract various recognition features (e.g., outline, color, features, size, etc.) to determine whether those features match the features of the learned object. The number of objects is unlimited, and the various views of those objects are also unlimited. Thus, training an AI model to recognize objects can be an ongoing process.
[0006] It is against this background that embodiments of the present disclosure emerge. Summary of the invention
[0007] Embodiments of the present disclosure relate to systems and methods for training AI models related to the playing process of a game application. The AI model is constructed by a deep learning engine, and the AI model is configured to provide various functionalities related to the playing process of a game application and / or a game application (e.g., providing recommendations, discovering the weaknesses of players, discovering defects in game applications, training players, providing opponents to players, etc.). The AI model can be trained using a large server network (e.g., a server console, a game server, etc.), and the server network is configured to execute an instance of the game application to support multiple playing processes. For example, the server network can support the playing process of a remote user, and each remote user plays the game application through a corresponding client device. In the normal process of supporting the playing process of a remote user, training data can be collected from the server network, and the training data can be used to train the AI model related to the game application. By collecting training data from an existing server network configured to support the playing process of a remote user, the time and energy consumed when collecting training data are concentrated and reduced. That is, the time required to construct an input training data set by recording on a server network is significantly reduced. To some extent, the AI model learns the complexity of the game application and how to play the game application. In this way, given the conditions of the game playing process of the game application (e.g., the game state of a specific game playing process) as input, the AI model can analyze the conditions and provide an appropriate response to the conditions. The response may depend on a predefined goal. Because the AI model understands how to play the game application given different goals (e.g., exploratory, direct to the end, easiest to play, hardest to play, etc.), the AI model can determine how to guide the game playing process (e.g., determine what the next input sequence required to achieve the corresponding goal is). Specifically, the AI model can be used to perform various functionalities related to the game application and / or the game playing process of the game application, wherein the functionality depends on the corresponding predefined goals. For example, the AI model can be used to train professional players to become the best of all players by having student players compete with ultimate opponents, or guiding student players to improve the weaknesses of student players through various challenges. In addition, the AI model can be continuously improved by continuously collecting training data and comparing new training data with existing training data based on success criteria.
[0008] In one embodiment, a method is disclosed, comprising: training an artificial intelligence (AI) model for a video game using training state data and associated success criteria collected from multiple game play sessions of a video game played by multiple players, wherein the AI model is configured to play the video game; receiving input state data during the players' game play sessions; applying the input state data to the AI model to determine a predicted degree of success for the game play sessions; and providing recommendations to the players' devices on how to play the video game based on the predicted degree of success.
[0009] In one embodiment, a non-transitory computer-readable medium is disclosed, storing a computer program for executing a method, the computer-readable medium comprising: program instructions for training an artificial intelligence (AI) model for a video game using training state data and associated success criteria collected from multiple game play sessions of a video game played by multiple players, wherein the AI model is configured to play the video game; program instructions for receiving input state data during the players' game play sessions; program instructions for applying the input state data to the AI model to determine a predicted degree of success for the game play sessions; and program instructions for providing recommendations to the players' devices on how to play the video game based on the predicted degree of success.
[0010] In one embodiment, a computer system is disclosed, comprising: a processor; a memory coupled to the processor and having instructions stored therein, which, if executed by the computer system, causes the computer system to perform a method for implementing a graphics pipeline, comprising: training an artificial intelligence (AI) model for a video game using training state data and associated success criteria collected from multiple game play sessions of a video game played by multiple players, wherein the AI model is configured to play the video game; receiving input state data during the player's game play session; applying the input state data to the AI model to determine a predicted degree of success of the game play session; and providing a recommendation to the player's device on how to play the video game based on the predicted degree of success.
[0011] In one embodiment, a method for AI training and application is disclosed, and more particularly, a method for processing an AI model for a gaming application is disclosed. The method includes training the AI model from multiple game play procedures of a context of the gaming application using training state data collected from the multiple game play procedures of the context and associated success criteria for each of the multiple game play procedures. The method includes receiving first input state data during a first game play procedure of the context. The method includes applying the first input state data to the AI model to generate an output indicating a degree of success of the context for the first game play procedure. The method includes performing an analysis of the output based on a predefined goal. The method includes performing an action based on the analyzed output to achieve the predefined goal.
[0012] In another embodiment, a non-transitory computer-readable medium storing a computer program for AI training and application is disclosed. The computer-readable medium includes program instructions for processing an AI model for a game application. The computer-readable medium includes program instructions for training the AI model from the multiple game play processes of the context of the game application using training state data collected from multiple game play processes of the context and associated success criteria for each of the multiple game play processes. The computer-readable medium includes program instructions for receiving first input state data during a first game play process of the context. The computer-readable medium includes program instructions for applying the first input state data to the AI model to generate an output indicating the degree of success of the context for the first game play process. The computer-readable medium includes program instructions for performing an analysis of the output based on a predefined goal. The computer-readable medium includes program instructions for performing an action based on the analyzed output to achieve the predefined goal.
[0013] In yet another embodiment, a computer system is disclosed, wherein the computer system includes a processor and a memory, the memory being coupled to the processor and having instructions stored therein, the instructions, if executed by the computer system, causing the computer system to perform the disclosed method for processing an AI model for a gaming application. The method performed by the computer system includes training the AI model from the multiple gaming processes of a context of the gaming application using training state data collected from the multiple gaming processes of the context and associated success criteria for each of the multiple gaming processes. The method includes receiving first input state data during a first gaming process of the context. The method includes applying the first input state data to the AI model to generate an output indicating a degree of success of the context for the first gaming process. The method includes performing an analysis of the output based on a predefined goal. The method includes performing an action based on the analyzed output to achieve the predefined goal.
[0014] In another embodiment, a method for AI training is disclosed. The method includes executing multiple instances of a game application at multiple servers, the multiple instances supporting multiple game play sessions of the game application. The method includes collecting training state data associated with corresponding game play sessions of a context of the game application at the multiple servers. The method includes defining success criteria for the training state data. The method includes training an AI model for the context by providing the training state data and the success criteria to a deep learning engine, wherein the trained AI model provides multiple outputs for multiple inputs.
[0015] Other aspects of the present disclosure will become apparent from the following detailed description taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The present disclosure may be best understood by referring to the following description in conjunction with the accompanying drawings, in which:
[0017] Figure 1A A system for training an artificial intelligence (AI) model through a back-end server network executing an instance of a gaming application according to one embodiment of the present disclosure, wherein the AI model is used to provide various functionalities related to the gaming application.
[0018] Figure 1B An exemplary neural network for building an AI model according to one embodiment of the present disclosure is shown.
[0019] Figure 2AA system diagram 200 for providing games through a cloud gaming network according to one embodiment of the present disclosure is shown.
[0020] Figure 2B A system according to one embodiment of the present disclosure is shown, the system providing games to multiple players of a gaming application executed through a cloud gaming network, and collecting training data from the game play of those players for training an AI model, wherein the AI model is configured to provide various functionalities related to the gaming application and / or the game play of the gaming application.
[0021] Figure 3A A method is shown according to one embodiment of the present disclosure for training an AI model by a back-end server network executing an instance of a game application, wherein the AI model is used to provide various functionalities related to the game application and / or the game playing process of the game application.
[0022] Figure 3B1 and Figure 3B-2 A data flow diagram according to one embodiment of the present disclosure is shown, which illustrates a process for training an AI model through a back-end server network executing an instance of a gaming application, wherein the AI model is used to provide various functionalities related to the gaming application and / or the gameplay process of the gaming application.
[0023] Figure 3C-1 is a screenshot of gameplay of a gaming application according to one embodiment of the present disclosure and illustrates one type of success criteria that may be used to train an AI model that understands how to play the gaming application.
[0024] Figure 3C-2 is an illustration of collecting training state data from multiple game play sessions, where multiple instances are executing on multiple servers to support the game play sessions, according to one embodiment of the present disclosure.
[0025] Figure 4A is a flow chart illustrating a method for applying an AI model configured to understand how to play a game application according to one embodiment of the present disclosure, wherein the AI model is used to provide various functionalities related to the game application and / or the game play process of the game application.
[0026] Figure 4B is a block diagram illustrating an AI processor engine configured for applying an AI model configured to understand how to play a game application, the AI model being used to provide various functionalities related to the game application and / or the game play process of the game application, according to one embodiment of the present disclosure.
[0027] Figure 5AA data flow diagram is shown according to one embodiment of the present disclosure, which illustrates a process for providing recommendations to players playing a game application, wherein the recommendations are provided by an AI model trained by a backend server network executing instances of the game application.
[0028] Figure 5B A data flow diagram is shown that illustrates a process for balancing a team of players playing a game application using player profiles determined by an AI model trained by a network of backend servers executing instances of the game application, according to one embodiment of the present disclosure.
[0029] Figure 5C A data flow diagram is shown according to one embodiment of the present disclosure, which illustrates the process of training an AI model through automated play guided by the AI model, where the AI model is trained through a backend server network executing instances of a gaming application.
[0030] Figure 5D A data flow diagram is shown according to one embodiment of the present disclosure, which illustrates the process of automatically sweeping a gaming application using an automated player guided by an AI model trained by a backend server network executing instances of the gaming application.
[0031] Figure 5E A data flow diagram is shown that illustrates a process of providing an opponent to a player, where the opponent is guided by an AI model trained by a network of backend servers executing instances of a gaming application, according to one embodiment of the present disclosure.
[0032] Fig. 5F A data flow diagram illustrating a process for identifying a player's weaknesses and providing various services to train the player to overcome those weaknesses is shown in accordance with one embodiment of the present disclosure.
[0033] Figure 6 Components of an exemplary apparatus are shown that may be used to perform aspects of various embodiments of the present disclosure. DETAILED DESCRIPTION
[0034] Although the following detailed description contains many specific details for illustrative purposes, those skilled in the art will appreciate that many variations and modifications of the following details are within the scope of the present disclosure. Therefore, in setting forth the various aspects of the present disclosure described below, the generality of the appended claims of this description is not compromised and no limitations are imposed on these claims.
[0035] In general, various embodiments of the present disclosure describe systems and methods for implementing deep learning (also referred to as machine learning) techniques to build AI models related to game applications and / or game application game play processes. In particular, the AI model is configured to provide various functionalities related to game applications and / or game application game play processes, including predicting and / or determining what actions to take in response to given conditions (e.g., game states) of the game application. For example, the AI model can be used to train professional players to become the best of all game players. The AI model is trained and / or constructed using a server network (e.g., through a cloud gaming system), and the server network executes game application instances that support one or more game play processes. The AI model can be trained based on success criteria, such as by following a path that is more successful than another similar path in terms of success criteria by the AI model. That is, the AI model learns to take a more successful path. In addition, the AI model can be continuously improved by continuously collecting training data, comparing new training data with existing training data, and selecting the best training data based on the success criteria.
[0036] With the above general understanding of the various embodiments in mind, exemplary details of the embodiments will now be described with reference to the various figures.
[0037] Figure 1A It is a system 100A for training an artificial intelligence (AI) model 160 by executing a back-end server network (e.g., a game cloud system) of an instance of a game application according to one embodiment of the present disclosure. According to one embodiment of the present disclosure, the AI model is used to provide various functionalities related to the game application and / or the game playing process of the game application. In particular, the trained AI model 160 can be implemented and / or supported by the back-end server through a network 150 (e.g., the Internet), wherein the back-end server provides artificial intelligence and / or deep machine learning (e.g., through a deep machine learning engine 190) for building and applying the trained AI model 160 related to the game application and / or the game playing process of one or more players in one or more locations around the world. The trained AI model 160 is trained to learn the complexity of the game application and / or the complexity of the game playing application (e.g., through the corresponding game playing process). In this way, given the conditions of the game playing process of the game application (e.g., the game state of a specific game playing process) as input, the AI model can analyze the conditions and provide an appropriate response to the conditions. The response can depend on a predefined goal (e.g., providing assistance, etc.). For example, the trained AI model 160 can determine how a gaming application should respond, or how a player should respond during a corresponding gameplay session.
[0038] As shown, multiple client devices 230 (e.g., devices 230a to 230n) are connected to a backend game cloud system (GCS) 201 via a network 150 (e.g., the Internet) to support multiple game play processes of a particular game application. For example, multiple remote players are playing a game application through corresponding client devices 230. The client device can be any type of computing device having at least a memory and a processor module, which is capable of connecting to a backend server system (e.g., GCS 201) via a network 150. The corresponding client device is configured to interact with an instance of the corresponding game application executed locally or remotely to implement the game play process of the corresponding player, such as through input commands for driving the game play process.
[0039] GCS 201 includes multiple servers 205 (e.g., servers 205a to 205n) that execute multiple game application instances 110 (e.g., instances 110a to 110n). For example, server 205a is configured to execute game application instance 110a to support corresponding players to perform corresponding game play processes through corresponding client devices. Each of the servers 205 can be configured to have at least a memory and a processor module, and the processor module can execute the game application, such as through the corresponding instance of the game application, to support the corresponding game play process. For example, each server 205 can be a server console, a game console, a computer, etc. Each server 205 is configured to stream data 105 (e.g., rendered images and / or frames corresponding to the game play process) back to the corresponding client device through the network 150. In this way, a computationally complex game application can be executed at the back-end server in response to a controller input received and forwarded by the corresponding client device. Each server can render images and / or frames, which are then encoded (e.g., compressed) and streamed to the corresponding client device for display.
[0040] In one embodiment, GCS 201 includes a distributed game engine system and / or architecture. In particular, the distributed game engine that executes the game logic is configured as a corresponding instance of a game application. In general, the distributed game engine has each function of the game engine, and those functions are assigned to multiple processing entities for execution. Each function can be further distributed in one or more processing entities. The processing entity can be configured with different configurations, including as metal or physical hardware, and / or as a virtual component or virtual machine, and / or as a virtual container, wherein the container is different from the virtual machine because it virtualizes the instance of the game application running on the virtualized operating system. The processing entity can utilize and / or rely on the server and its basic hardware on one or more servers (computing nodes) of GCS 201, wherein the server can be located on one or more racks. The coordination, assignment and management of the execution of those functions assigned to various processing entities are performed by the distribution synchronization layer. In this way, the execution of those functions is controlled by the distribution synchronization layer, in response to the controller input of the player, so that media (e.g., video frames, audio, etc.) can be generated for the game application. The distributed synchronization layer enables those functions to be efficiently performed (eg, via load balancing) across distributed processing entities, such that key game engine components / functions are distributed and reorganized for more efficient processing.
[0041] These various functions performed by the game engine include basic processor-based functions for executing game applications and services associated with game applications. For example, processor-based functions include 2D or 3D rendering, physics, physics simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, ray tracing, shadows, culling, transformation, artificial intelligence, etc. In addition, the services of the game application include streaming, encoding, memory management, multi-threading management, quality of service (QoS), bandwidth testing, social networking, management of social friends, communication with friends' social networks, communication channels, texting, instant messaging, chat support, etc.
[0042] In addition, by adding more processing entities, the distributed synchronization layer can be easily scaled (on a separate distributed game engine) to handle increased processing requirements or processing complexity. That is, processing power can be elastically increased or decreased based on demand. In addition, the distributed game engine system can be scaled across many users, each supported by a corresponding distributed game engine, such as in a multi-player game session of a game application. Thus, a multi-player game session is not constrained by hardware limitations, which limit the maximum number of players in a session due to performance issues (e.g., latency). In contrast, a scalable distributed game engine system is able to expand or shrink the number of distributed game engines based on the number of players participating in a multi-player game session without any hardware constraints. In this way, tens of thousands of players can participate in a single multi-player game session.
[0043] In system 100A, AI model 160 can be built (e.g., trained) using a server network that is configured to execute instances of a game application to support multiple game play processes. In particular, training data 345 is collected from the execution of game application instances at multiple servers 205. In particular, the collection of training data is performed without knowing any remote players associated with various game play processes of the game application. Training data 345 includes controller input 236 for guiding corresponding game play processes, game state 133 of the game application instance that occurs during the game play process, response metrics, success criteria 135, success metrics, additional analysis performed on the game play process, etc. In some cases, training data 345 can be generated by an instance of the game application (e.g., game state metadata, etc.). Training data 345 is provided as input to a deep learning and / or machine learning engine 190. Deep learning engine 190 includes a modeler 120 that is configured to use training data 345 to build and / or train AI model 160, as described below with respect to Figure 1B and Figure 3A , Figure 3B-1 and Figure 3B-2 Further described.
[0044] The AI model 160 may be implemented (e.g., by a player, by an automated player, etc.) during a subsequent game play process of a game application (e.g., after training of the AI model). For example, the AI model may be implemented and / or executed at a backend server to support a game play process, where the game play process may be executed on a local device (for a player) or at a backend server. For example, the trained AI model is implemented by a deep learning engine 190 to provide various functionality to the game play process of a game application.
[0045] As shown, the analyzer 140 is configured to utilize an AI model 160, which is trained to provide various functionalities related to the game playing process of the game application. In particular, the input data stream 405 is provided as an input to the deep learning engine 190, which is configured to implement the trained AI model 160. The trained AI model 160 provides an output in response to the input, wherein the output depends on the predefined functionality and / or predefined goals of the trained AI model 160. For example, the trained AI model 160 can be used by the analyzer 140 to determine what actions need to be taken during the game playing process-taken by the player, or taken by the corresponding execution instance of the game application. The analyzer 140 includes an action generator 170, which is configured to perform actions in response to the input state data 405 and taking into account the predefined goals of the trained AI model 160. In this way, the analyzer can provide various functionalities by using the AI model 160, including providing services to players playing the game application (e.g., providing recommendations, discovering player weaknesses, training players, providing opponents to players, discovering flaws in the game application, etc.).
[0046] More particularly, the analyzer 140 is configured to perform various functionalities related to the game application and / or the game play process of the game application. The analyzer 140 is configured to analyze the output from the trained AI model 160 for a given input (e.g., controller input 236, game state data 133, success criteria 135) and provide a response (e.g., an action). For example, the analyzer 140 can provide a profile of a player playing the game application; provide recommendations to the player during the player's game play process of the game application, wherein the recommendations can be structured with consideration of the user profile; take over the game play process; use the player profile to build a fairly balanced team (e.g., a team that competes against each other in the game application); automatically play the game application, such as for the purpose of automatically training the AI model; provide a robot opponent; explore the game application; determine the weaknesses of the corresponding player and perform services to help the player overcome the weaknesses, etc.
[0047] Figure 1BAn exemplary neural network 190 (e.g., a deep learning engine) is shown for building a trained AI model that partially learns the complexity of a gaming application and how to play the corresponding gaming application. In this way, given a condition of a gaming application's gameplay (e.g., a game state of a particular gaming application) as input, the AI model can analyze the condition and provide an appropriate response to the condition. For example, the AI model can be used to provide various functionalities related to the gaming application and / or the gaming application's gameplay, including predicting and / or determining what action to take in response to a given condition (e.g., a game state) of the gaming application.
[0048] More particularly, the AI model is trained and / or built using a server network (e.g., via a cloud gaming system) that executes a gaming application instance that supports one or more game play processes. According to one embodiment of the present disclosure, the training is implemented by a deep learning engine 190. In one embodiment, the neural network 190 can be implemented within an AI processor or engine 210 of a backend server. In particular, Figure 1A The modeler 120 of the deep learning engine 190 in the system 100A is configured to learn everything about the gaming application for use by any player (e.g., real or virtual) during subsequent gameplay of the gaming application.
[0049] Specifically, the deep learning or machine learning engine 190 cooperates with the modeler 120 and is configured to analyze the training data 345 collected at multiple back-end servers, which are configured to execute multiple instances of the game application. The deep learning engine 190 uses artificial intelligence, including deep learning algorithms, reinforcement learning, or other artificial intelligence-based algorithms to build a trained AI model related to the game application and / or the game application's game play process. In this way, a large amount of training data can be effectively collected, in part to help define the game application and / or any game play process of the game application, including the scenarios in which the game application can be presented (e.g., different scenarios, and parts within those scenarios, as defined in the trained AI model). In addition, the AI model can be used (e.g., during or after training, such as by the analyzer 140 of the AI processor engine 210) to provide various functionalities related to the game application and / or the game application's game play process in any subsequent game play process of the game application. Thus, the deep learning engine 190 is able to learn everything about the game application and / or the game play process of the game application, so that the trained AI model can be used to provide the best service for the subsequent game play process of the game application, such as providing it to the player. For example, the trained AI model can be used to train professional game players to become the best players in the world, or to provide various tutorial sessions (e.g., video tutorials, game play challenges, etc.) designed to address player weaknesses. The deep learning engine 190 can be configured to continuously improve the trained AI model given any updated training data. Improvement is based on determining which groups of training data to use for training, and the determination is based on the performance of those groups of training data in the game application, and the performance is based on the corresponding success criteria.
[0050] More specifically, during the learning and / or modeling phase, the deep learning engine 190 uses training data 345 to predict and / or determine how successful a particular game play process of a game application will be given a set of input data. For a given game play process of a game application given the set of input data, the resulting AI model of the game application can be used to determine the action to be performed. That is, the trained AI model can be used (e.g., by the analyzer 140) to provide various functionalities related to the game application and / or the game play process of the game application. For example, the input data can be game state data (e.g., including controller input, etc.), and the trained AI model 160 can be used to generate a response to the input data. The user can provide a response with or without any request or knowledge.
[0051] For example, the trained AI model 160 can be used by the analyzer 140 to provide recommendations to players playing game applications. The trained AI model 160 can be used by the analyzer 140 to create a user profile of a player that is specific to a game application or a context of a game application. The AI model 160 can be used by the analyzer 140 to control a robot opponent within a game application, such as when training a player against the world's best opponent created by training data, when training a player against a virtual opponent, so that the user can gradually become better (and constantly be improved through the self-game) through self-game (e.g., playing against a virtual version of himself or herself). The AI model 160 can be used by the analyzer 140 to discover the player's weaknesses and provide tutorial sessions to address those weaknesses. The AI model 160 can be used by the analyzer 140 to discover problems within the game application (e.g., code vulnerabilities that cause glitches, etc.). Other functionality is also supported, although not necessarily described.
[0052] Neural network 190 represents an example of an automatic analysis tool for analyzing a data set to determine the complexity of a game playing application, including responses and / or actions that can be determined and / or performed during the game playing process of the game application. Different types of neural networks 190 are possible. In an example, neural network 190 supports deep learning that can be implemented by deep learning engine 190. Thus, deep neural networks, convolutional deep neural networks, and / or recurrent neural networks using supervised or unsupervised training can be implemented. In another example, neural network 190 includes a deep learning network that supports reinforcement learning or reward-based learning (e.g., by using success criteria, success metrics, etc.). For example, neural network 190 is configured as a Markov decision process (MDP) that supports a reinforcement learning algorithm.
[0053] In general, neural network 190 represents a network of interconnected nodes, such as an artificial neural network. Each node learns some information from the data. Knowledge can be exchanged between nodes through the interconnection. The input of neural network 190 activates a set of nodes. In turn, this set of nodes activates other nodes, thereby spreading knowledge about the input. This activation process is repeated on other nodes until an output is provided.
[0054] As shown, neural network 190 includes a hierarchy of nodes. At the lowest level of the hierarchy, there is an input layer 191. Input layer 191 includes a set of input nodes. For example, each of these input nodes maps to an instance of a game play process of a game application, wherein these instances include one or more features defining the instance (e.g., controller input, game state, result data, etc.). The intermediate predictions of the model are determined by a classifier that creates a label (e.g., output, feature, node, classification, etc.).
[0055] At the highest hierarchical level, there is an output layer 193. The output layer 193 includes a set of output nodes. For example, the output node represents a decision related to one or more components of the trained AI model 160 (e.g., for a given set of input data, an action, a prediction, a prediction of the success of the game play process, etc.). As previously described, the output node can identify a predicted or expected action or a learned action for a given set of inputs, where the input can define various situations or parts of the situation of the game application. These results can be compared with predetermined and real results, or compared with learned actions and results obtained from the game play process for collecting training data, so as to improve and / or modify the parameters used by the deep learning engine 190, so as to iteratively determine the appropriate prediction or expected response and / or action for a given set of inputs. That is, the nodes in the neural network 190 learn the parameters of the trained AI model 160, and the AI model is used to make such decisions when improving the parameters.
[0056] In particular, hidden layer 192 exists between input layer 191 and output layer 193. Hidden layer 192 includes "N" hidden layers, where "N" is an integer greater than or equal to 1. In turn, each hidden layer also includes a set of hidden nodes. The input nodes are interconnected to the hidden nodes. Similarly, the hidden nodes are interconnected to the output nodes, so that the input nodes are not directly interconnected to the output nodes. If there are multiple hidden layers, the input nodes are interconnected to the hidden nodes in the lowest hidden layer. In turn, these hidden nodes are interconnected to the hidden nodes in the next hidden layer, and so on. The hidden nodes in the next highest hidden layer are interconnected to the output nodes. The interconnection connects two nodes. The interconnection has a numerical weight that can be learned so that the neural network 190 adapts to the input and is able to learn.
[0057] Typically, the hidden layer 192 allows knowledge about the input nodes to be shared between all tasks corresponding to the output nodes. To this end, in one implementation, a transformation f is applied to the input nodes by the hidden layer 192. In the example, the transformation f is nonlinear. Different nonlinear transformations f are available, including, for example, a rectification function f(x)=max(0,x).
[0058] The neural network 190 also uses a cost function c to find an optimal solution. For a given input x, the cost function measures the deviation between the prediction output by the neural network 190, defined as f(x), and the ground truth or target value y (e.g., the expected result). The optimal solution represents a situation where no solution has a cost lower than the cost of the optimal solution. For data for which ground truth labels are available, an example of a cost function is the mean squared error between the prediction and such ground truth. During the learning process, the neural network 190 can use a back-propagation algorithm to employ different optimization methods to learn the model parameters (e.g., the weights of the interconnections between the nodes in the hidden layer 192) that minimize the cost function. An example of such an optimization method is stochastic gradient descent.
[0059] In an example, the training data set for the neural network 190 can be from the same data domain. For example, the neural network 190 is trained to learn a predicted or expected response and / or action to be performed for a given set of inputs or input data. In the illustrated description, the data domain includes game play data collected through multiple game play sessions of multiple users to define baseline input data. In another example, the training data set is from a different data domain, including input data other than the baseline.
[0060] Thus, the neural network 190 may partially predict or determine predicted or expected responses and / or actions to be performed for a given set of inputs (conditions of the gaming application, such as the game state). Based on these predictions, the neural network 190 may also define a trained AI model 160 that is used to provide the determination of those results and / or actions to be performed given a set of inputs (e.g., various functionalities related to the gaming application and / or the game play process of the gaming application).
[0061] Figure 2AA system 200A is shown that supports the collection of training data for building and / or training an AI model associated with a game application and / or a game application's game play process according to an embodiment of the present disclosure. In particular, system diagram 200A allows access to and play of video games stored in a game cloud system (GCS) 201. In general, a game cloud system GCS 201 may be a cloud computing system that operates over a network 220 to support multiple players playing a game application through a corresponding game play process. Data associated with those game play processes may be provided as training data, which is used to build and / or train an AI model associated with a game application and / or a game application's game play process. In particular, system 200A includes GCS 201, one or more social media providers 240, and client devices 230, all of which are connected via a network 150 (e.g., the Internet). One or more user devices may be connected to network 150 to access services provided by GCS 201 and social media providers 240.
[0062] In one embodiment, the game cloud system 201 includes a game server 205, a recorder 271, a tag processor 273, and an account manager 274 including a user profile manager, a game selection engine 275, a game session manager 285, a user access logic 280, a network interface 290, and a social media manager 295. The GCS 201 may also include multiple game storage systems, such as a game state storage area, a random seed storage area, a user-saved data storage area, and a snapshot storage area, which can generally be stored in the data storage area 260. Other game storage systems may include a game code storage area 261, a recorded game storage area 262, a tag data storage area 263, a video game data storage area 264, and a game network user storage area 265. In one embodiment, the GCS 201 is a system that can provide game applications, services, game-related digital content, and interconnectivity between systems, applications, users, and social networks. The GCS 201 can communicate with a user device 230 and a social media provider 240 via a network interface 290 through a social media manager 295. The social media manager 295 can be configured to associate one or more friends. In one embodiment, each social media provider 240 includes at least one social graph 245 showing the user's social network connections.
[0063] The player / user 5 can access the services provided by the GCS 201 via the game session manager 285. For example, the account manager 274 enables the player 5 to authenticate and access the GCS 201. The account manager 274 stores information about the member users / players. For example, the user profile of each member user can be managed by the account manager 274. In this way, the account manager 274 can use the member information for authentication purposes. For example, the account manager 2274 can be used to update and manage user information related to the member users. In addition, the game titles owned by the member users can be managed by the account manager 274. In this way, the video games stored in the data storage area 264 are made available to any member user who owns those video games.
[0064] In one embodiment, a user, such as player 5, can access services provided by GCS 201 and social media provider 240 via a client device 230 through a connection on network 150. Client device 230 may include any type of device having a processor and memory, wired or wireless, portable or non-portable. In one embodiment, client device 230 may be a smartphone, tablet computer, or a hybrid that provides touch screen capabilities in a portable form factor. An exemplary device may include a portable telephone device that runs an operating system and has access to various applications (apps) that can be obtained through network 150 and executed on a local portable device (e.g., smartphone, tablet computer, laptop computer, desktop computer, etc.).
[0065] The client device 230 includes a display 232 that acts as an interface for the player 5 to send input commands 236 and to display data and / or information 235 received from the GCS 201 and social media providers 240. The display 232 may be configured as a touch screen, or a display typically provided by a flat panel display, cathode ray tube (CRT), or other device capable of rendering a display. Alternatively, the client device 230 may have its display 232 separate from the device, similar to a desktop computer or laptop computer.
[0066] In one embodiment, the client device 230 is configured to communicate with the GCS 201 to enable the player 5 to play a video game. For example, the player 5 can select (e.g., by game title, etc.) a video game available in the video game data storage area 264 via the game selection engine 275. In this manner, the selected video game is enabled and loaded for execution by the game server 205 on the GCS 201. In one embodiment, the game play process is primarily performed in the GCS 201, such that the client device 230 will receive a stream of game video frames 235 from the GCS 201, and user input commands 236 for driving the game play process are transmitted back to the GCS 201. The video frames 235 received from the streamed game play process are displayed in the display 232 of the client device 230.
[0067] In one embodiment, after the player 5 selects an available game title to play, the player 5 may initiate a game session for the selected game title through the game session manager 285. The game session manager 285 first accesses the game state storage area in the data storage area 140 to retrieve the saved game state (for the selected game) of the last session played by the player 5, if any, so that the player 5 can restart the game play process from the point where the previous game play process stopped. Once the resume point or starting point is identified, the game session manager 285 may notify the game execution engine in the game processor 210 to execute the game code of the selected game title from the game code storage area 261. After initiating the game session, the game session manager 285 may transmit the game video frames 235 (i.e., streaming video data) to a client device, such as the client device 230, via the network interface 290.
[0068] During the game play process, the game session manager 285 can communicate with the game processor 210, the recording engine 271 and the tag processor 273 to generate or save a record (e.g., video) of the game play process or the game play process session. In one embodiment, the video record of the game play process can include the tagged content entered or provided during the game play process, as well as other game related metadata. The tagged content can also be saved via snapshots. The video record of the game play process and any game metrics corresponding to the game play process can be saved in the recorded game storage area 262. Any tagged content can be saved in the tag data storage area 263.
[0069] During the game play process, the game session manager 285 can communicate with the game processor 204 to deliver and obtain user input commands 236, which are used to affect the outcome of the corresponding game play process of the video game. The input commands 236 entered by the player 5 can be transmitted from the client device 230 to the game session manager 285 of the GCS 201. The input commands (e.g., controller inputs) 236 including input commands to drive the game play process can include user interactive inputs, such as including markup content (e.g., text, images, video recording clips, etc.). The game input commands and any user game metrics (how long the user plays the game, etc.) can be stored in the game network user storage area. The selection information related to the game play process of the video game can be used to implement multiple features available to the user.
[0070] Figure 2B A system 200B is shown that provides for the collection of training data for building and / or training an AI model 160 associated with a game application and / or a game application's game play process according to one embodiment of the present disclosure, wherein an instance and / or instantiation of a game application supporting the game play process is executed via a cloud gaming network. In addition, the system 200A is configured to support the implementation of an AI model 160 that is trained to provide various functionalities associated with the game application and / or the game application's game play process (e.g., providing recommendations, training players, discovering player weaknesses, providing robot opponents, etc.).
[0071] As shown, according to one embodiment of the present disclosure, system 200B provides game control to multiple players 215 (e.g., players 5L, 5M ... 5Z) playing game applications, which are executed through a cloud game network. In some embodiments, the cloud game network can be a game cloud system 210 including multiple virtual machines (VMs) running on a hypervisor of a host machine, wherein one or more virtual machines are configured to utilize the hardware resources available to the hypervisor of the host to execute a game processor module. In other embodiments, GCS 201 includes a distributed game engine system and / or architecture for executing game logic, and the game logic is configured as a corresponding instance of a game application. In general, a distributed game engine has each function of a game engine, and those functions are assigned to multiple processing entities on one or more servers of GCS 201 for execution. Each function can be further distributed in one or more processing entities. Now with reference to the accompanying drawings, the same reference numerals designate the same or corresponding parts.
[0072] As shown, the game cloud system 210 includes a game server 205 providing access to multiple interactive video games or game applications.Game server 205 can be any type of server computing device available in the cloud, and can be configured as one or more virtual machines executed on one or more hosts.For example, game server 205 can manage the virtual machine supporting a game processor, and the game processor instantiates the instance of the game application for the user.Thus, the multiple game processors of the game server 205 associated with multiple virtual machines are configured to perform multiple instances of the game application associated with the game playing process of multiple users 215.In this way, the back-end server supports the media stream (for example, video, audio, etc.) of the game playing process of multiple game applications provided to multiple corresponding users.As will be further described below, the training data collected from the multiple game processors executing the instance of the game application are used to build and / or train the AI model related to the game playing process of the game application and / or the game application.
[0073] A plurality of players 215 access the gaming cloud system 210 via the network 150, wherein the players (eg, players 5L, 5M ... 5Z) access the network 150 via corresponding client devices 230'. The client devices 230' may be similarly configured as Figure 1A The client device 230 of the corresponding player 5L may be configured as a thin client, which provides an interface with a back-end server providing computing functionality. In particular, the client device 230' of the corresponding player 5L is configured to request access to the game application through a network 150 such as the Internet, and is used to render an instance of the game application (e.g., a video game) executed by the game server 205 and delivered to the display device associated with the corresponding player 5L. For example, the player 5L can interact with the instance of the game application executed on the game processor of the game server 205 through the client device 230'. More particularly, the instance of the game application is executed by the game title execution engine 211 (e.g., a game engine), which runs the game logic 177 corresponding to the game application. The game logic (e.g., executable code) 177 that implements the game application is stored and accessed through the previously described data storage area 260 or the game code storage area 261 or the video game storage area 264, etc., and is used to execute the game application. As shown in the figure, the game title processing engine 211 can use multiple game logics 177 to support multiple game applications.
[0074] As previously described, the client device 230' may receive input from various types of input devices 11, such as a game controller, a tablet computer, a keyboard, gestures captured by a camera, a mouse touchpad, etc. The client device 230' may be any type of computing device having at least a memory and a processor module, which is capable of connecting to the game server 205 via the network 150. Moreover, the corresponding player's client device 230' is configured to generate a rendered image executed by the remotely executed game title execution engine 211, and to display the rendered image on a display including a head mounted display (HMD) 102. For example, the corresponding client device 230' is configured to interact with the remotely executed instance of the corresponding game application to implement the corresponding player's game play process, such as through input commands for driving the game play process.
[0075] In another embodiment, the previously described multi-player processing engine 119 provides control of a multi-player gaming session for a gaming application. In particular, when the multi-player processing engine 119 is managing a multi-player gaming session, the multi-player session controller 116 is configured to establish and maintain a communication session with each of the users and / or players in the multi-player session. In this manner, the players in the session can communicate with each other under the control of the multi-player session controller 116.
[0076] In addition, the multi-player processing engine 119 communicates with the multi-player logic 118 to enable interaction between players within the corresponding game environment of each player. In particular, the state sharing module 117 is configured to manage the state of each of the players in the multi-player game session. For example, the state data may include game state data, which defines the state of the game process (of the game application) of the corresponding player (e.g., player 5L) at a specific point, as previously described. In addition, the state data may include data saved by the user / player, which includes information that personalizes the video game of the corresponding player, as previously described. For example, the state data includes information associated with the user's character, so that the video game is presented with a character (e.g., appearance, appearance, clothing, weapons, etc.) that may be unique to the user. In this way, the multi-player processing engine 119 using the state sharing data 117 and the multi-player logic 118 can overlay / insert objects and characters into each of the game environments of the users participating in the multi-player game session. This allows the users in the multi-player game session to interact via each of their corresponding game environments (e.g., as displayed on the screen).
[0077] In addition, backend server support via the AI processor 210 (which may be integrated within the GCS 201 or located remotely from the GCS 201) may provide for the construction and / or training of the AI model 160 associated with the game application and / or the game application's game play, and also provide implementations and / or applications of the AI model 160. In particular, the backend AI processor 210 includes the previously described deep learning engine 190, which is configured to partially learn and / or model responses and / or actions (e.g., controller inputs, etc.) to be performed for a given set of any inputs (e.g., inputs defining conditions for the game application's game play, the conditions including game states, etc.) in order to build (e.g., via the modeler 120) and apply (e.g., via the analyzer 140) the trained AI model associated with the game application and / or the subsequent game play of the game application. For example, the modeler 120 within the deep learning engine 190 can be operated to set parameters defined within the deep learning engine 190, which define various nodes in the input layer 191, the hidden layer 192, and the output layer 193 for applying the trained AI model 160 within the deep learning engine 190. As previously described, the modeler 120 can set the parameters in the AI model 160 based on one or more success criteria used during training. In this way, the AI model 160 is trained to learn the complexity of the game application and / or the complexity of playing the game application, so that the AI model 160 can be used to provide various functionalities related to the game application and / or the game playing process of the game application (e.g., predicting and / or determining what actions to take in response to given conditions such as game states, including controller inputs). Thus, the analyzer 140 is capable of analyzing the output from the AI model 160 in response to the input of the conditions defining the game playing process, and optionally analyzing the input (e.g., input state data) to provide an appropriate response to the condition, wherein the response can depend on a predefined goal (e.g., providing assistance, providing guidance, etc.).
[0078] According to one embodiment of the present disclosure, through a detailed description of various modules of system 100A and system 200B, flowchart 300A discloses a method for training an AI model by executing a backend server network of an instance of a game application, wherein the AI model is used to provide various functionalities related to the game application and / or the game playing process of the game application. As previously described, flowchart 300A can be implemented in a backend server (e.g., implemented in combination with deep learning engine 190 in game cloud system 201).
[0079] In particular, at 302, the method is included in multiple instances of executing game applications on multiple servers, and the multiple instances support multiple game playing processes of game applications. For example, multiple servers can operate in the game cloud system (for example, GCS 201) described previously, wherein the game application is remotely executed from the corresponding player, and media (for example, video frames, audio, etc.) is streamed to the client device of the player through the network. In this case, multiple game playing processes of the game application are controlled by multiple players via corresponding client devices, and the client device is away from the server. In other implementations, the game application is executed locally by the user / player, and the metadata from the game application being executed is delivered to the back-end server through the network, for the purpose of analysis (for example, for the purpose of training AI model), or for supporting multi-player game sessions. In other embodiments, multiple game playing processes (for example, by AI) can be automatically controlled (for example, for self-training AI model).
[0080] At 304, the method includes collecting training state data associated with corresponding game play processes of the context of the game application at multiple servers. The training state data may include metadata associated with the game play process, including controller input, game state, progress of the game play process, results of the context (e.g., success or failure), user profile information, etc. In other words, the training state data includes any data that may be relevant to understanding the game application and / or the game play process of the game application. Because the instance of the game application is executed at the back-end server, access to the training state data can be easily obtained without the active participation of the associated players of those game play processes. In other words, the players may not even know that the training state data is collected.
[0081] The training state data may include game state data, which defines the state and / or condition of the game application at a specific point (e.g., during the game play process). For example, the game state data may include game characters, game objects, game object attributes, game attributes, game object states, graphic overlays, etc. In this way, the game state data allows the generation of a game environment at a corresponding point in the video game. The game state data may also include the state of each device used to render the game play process, such as the state of the CPU, GPU, memory, register values, program counter values, programmable DMA states, DMA buffer data, audio chip states, CD-ROM states, etc. The game state data may also identify which parts of the executable code need to be loaded from then on to execute the video game. Not all game state data need to be captured and stored, just enough to allow the executable code to start the game at the point corresponding to the snapshot.
[0082] In addition, the training state data may include user-saved data that personalizes the corresponding user's game application. This includes information associated with the user's character so that the video game is presented with a character that may be unique to the user (e.g., appearance, appearance, clothing, weapons, etc.). In this way, the user-saved data enables the generation of a character for the corresponding user's game play process, wherein the character has a state corresponding to the point associated with the snapshot in the video game. For example, the user-saved data may include the game difficulty, game level, character attributes, character position, number of lives remaining, possible total number of lives available, armor, trophies, time counter values, etc. selected by the corresponding player when playing the game application. The user-saved data may also include user profile data that identifies the corresponding player.
[0083] In addition, the training state data may also include random seed data generated by AI. The random seed data may not be part of the original game code, but may be added to the overlay to make the game environment look more real and / or more attractive to the user. That is, the random seed data provides additional features for the game environment at the corresponding point in the player's game play process. For example, AI characters can be randomly generated and provided in the overlay. AI characters are placed in the game environment to enhance the user experience and may or may not affect the game play process. As an illustration, these AI characters may walk randomly in a city scene. In addition, other objects can also be generated and presented in the overlay. For example, clouds in the background and birds flying in space can be generated and presented in the overlay.
[0084] The game application may include one or more scenarios. A scenario may be a critical point in the game application (e.g., necessary to advance in the game application), such as fighting a boss at the end of a level, or jumping over an object or obstacle that blocks the only path to the destination, or crossing an obstacle (e.g., climbing a mountain, crossing a lake or river with aggressive crocodiles, etc.). A scenario may be less critical, such as when completing an intermediate task in a game application. In these cases, the scenario may involve completing a task to obtain a reward (e.g., money, an important sword, etc.). The data collected for training the AI model may be limited to data related to the game play process during one or more scenarios of interest. In this way, the trained AI model understands the complexity of playing the game application using data related to the scenario during the scenario, and there is no pollution from data that may be irrelevant to playing the scenario.
[0085] The situation may be predefined, such as by a developer of a game application. For example, the situation may be designed to be of high difficulty, such that it is expected that many players will not be able to advance through the situation. In other cases, the situation may be found by analyzing the collected training state data. That is, it may become clear that a particular portion of the game application is difficult for the player to advance during their corresponding game play. In this case, the portion of the game application may be identified as a situation, such that training state data is collected during the game play of the identified situation.
[0086] At 306, the method includes defining a success criterion for training state data. For example, the success criterion can be used by a deep learning engine for the purpose of training an AI model. Specifically, when an AI model (e.g., perfecting the weights of the mutual relationship between two nodes defining different layers, etc.), the success criterion can be applied to define the mutual relationship between nodes of multiple layers. For example, the success criterion can be used to distinguish similar training state data and give some insights into how to play or choose to play a game application during a scenario. In a simplified example, a scenario may involve completing a task (e.g., defeating a boss, crossing an obstacle, etc.), wherein two sets of training data each describe or outline the process of how to play in the scenario (e.g., controller input, strategy, etc.). The success criterion can be used to determine which set of training state data is more successful in completing the task. In order to train an AI model related to the scenario of a game application and / or the game playing process of the scenario of a game application, a more successful set of training state data can be more weighted than another less successful set of training state data (e.g., when defining the mutual relationship between nodes of an AI model).
[0087] At 308, the method includes training an AI model for the situation by providing training state data and success criteria to a deep learning engine, wherein the trained AI model provides multiple outputs for multiple inputs. Because the training state data is related to the situation, the AI model is trained to learn everything about the situation in the game application and / or the game process of the situation based on one or more success criteria. In this way, when the AI model is given a set of inputs (e.g., game state, controller input, etc.) related to the subsequent game process of the situation, the AI model can provide outputs that may be beneficial to the game process. In one embodiment, the output can indicate the degree of success of the game process. That is, given the current conditions of the game process (e.g., game state), the AI model can predict the direction of the game process and predict how successful the game process is in the corresponding situation. Additional analysis of the output (e.g., by analyzer 140) and analysis of the group input (e.g., current and past multiple groups of inputs) can provide a response to the group input. In particular, actions can be performed as outputs and in response to the group input. For example, if the set of inputs indicates that the game play process is being directed in a direction that is not successfully advancing in the scenario (e.g., via the output of the AI model), the output from the AI model can be used (e.g., by the analyzer 140) to provide recommendations or suggestions on how to advance through conditions encountered during the game play process of the scenario of the game application. If the game play process is associated with a remote player, the recommendations can be provided without any prompting from the remote player, so that the set of inputs is automatically provided to the trained AI model during the game play process. The recommendations can help the player successfully advance in the scenario, advance in the scenario effectively, win the most assets playing the scenario, complete the tasks in the scenario that give the player the best chance of advancing in the scenario at a later stage in the game application, and so on. In other embodiments, the recommendations are provided at the request of the remote player, so that in response to the request, the set of inputs is provided to the AI model.
[0088] Figure 3B-1 and Figure 3B-2 A data flow diagram is shown according to one embodiment of the present disclosure, which illustrates a process for training an AI model by executing a backend server network of instances of a gaming application. As previously described, the AI model learns about the gaming application and / or the gaming application's gameplay, such that given input (e.g., input related to the gaming application's conditions - game state, etc.), the AI model can be used to provide various functionalities related to the gaming application and / or the gaming application's gameplay (e.g., providing assistance, etc.). Figure 3B-1 Compared to the functionality performed by the modeler 120, Figure 3B-2The process provides different functionality performed by the modeler 120', as will be described below.
[0089] like Figure 3B-1 and Figure 3B-2 As shown in, multiple game play processes 310 (e.g., 310a to 310n) of a game application are shown. Instances of the game application can be executed to support the game play process, wherein these instances are executed on a back-end server, such as a game cloud system, as previously described. The game application can include one or more interesting situations. As shown, the game application includes a first situation (S-1), a second situation (S-2), and a third situation (S-3). For example, a situation may be rated as difficult by a developer, or discovered through the game play process of the game application. In other examples, a situation may be an interesting part of a game application that is popular with players, so that these players may be eager to learn everything about the situation.
[0090] In one embodiment, multiple game play processes 310 are controlled by multiple players P-1 to pn through corresponding client devices. In another embodiment, multiple game play processes 310 can be automatically controlled, such as in order to use multiple back-end servers to self-train AI models. As shown in the figure, the game play process provides various game play process data 320a to 320n. The game play process data can include metadata, including game state data, as previously described. For example, the game state data describes the state of the game at a specific point, and can include controller input data. In addition, the game play process data 320a to 320n can include a record of the game play process 310a to 310n, for extracting metadata and / or training state data.
[0091] like Figure 3B-1 and Figure 3B-2 As shown, the capture engine 340 captures game play data 320a to 320n, as well as other data that can be provided, such as success criteria 330. As previously described, the success criteria can be used to distinguish similar training state data for the purpose of training the AI model. In particular, the success criteria are used to train the AI model, including defining the interrelationships between the nodes of each layer within the AI model, where the nodes can define features related to the game application and / or the game application's game play process (e.g., controller input, game state, etc.). In the example, the success criteria can be used to determine which set of training state data is more successful, such as defining the weights of one or more paths (between nodes of one or more layers) through the AI model. In this way, the AI model can be used to give insights (e.g., strategies) on how to play or choose how to play the game application during the scenario. The captured data is provided to the deep learning engine 190 as training state data 345.
[0092] like Figure 3B-1 and Figure 3B-2 As shown in , the deep learning engine 190 includes a modeler 120 configured to train and / or build an AI model using training state data 345 based on one or more success criteria. The modeler 120 can implement artificial intelligence through various neural networks (e.g., convolutional, recursive, etc.). The modeler 120 can implement AI through various algorithms, including (for illustration, as examples) deep learning, reinforcement learning, supervised learning, unsupervised learning, reward-based learning (e.g., by using success criteria, success metrics, etc.), incremental learning, etc. The deep learning engine 190 was previously described in Figure 1B In particular, when building an AI model, the modeler 120 identifies a set of feature-related rules that make predictions given a set of inputs (e.g., features that can define a situation or condition—a game state—of a gaming application). The predictions can include how successful a given set of inputs can be when playing a situation. In this way, the AI model can be used to determine what actions to take given a set of inputs.
[0093] like Figure 3B-1 and Figure 3B-2 As shown in , the modeler 120 of the deep learning engine 190 includes a feature recognition engine 350 configured to recognize multiple features of the training state data. For each game play process corresponding to the scenario, the training state data includes features. For example, at a specific point in the game play process, an instance of the training state data can be collected, where the training instance includes one or more features (e.g., a set of features for the training instance), where the features can include variables, parameters, controller inputs, game state metadata, etc.
[0094] Thus, the feature recognition engine 350 is configured to parse the training state data in order to identify and / or extract features from the data. The feature recognition engine 350 may also be configured to learn features. At each training cycle through the deep learning engine 190, a training instance (e.g., a set of features) is provided as input, where the training instance may be associated with a specific point in the game play of a situation. In this way, the deep learning engine 190 is configured to incrementally learn the game application, the situation of the game application, and / or the game play of the situation of the game application.
[0095] like Figure 3B-1As shown in , the modeler 120 is configured to learn rules that define the relationship between features and outputs (e.g., predictions, actions, etc.), where features can be defined within one or more nodes located at one or more hierarchical levels of the trained AI model 160. The modeler 120 builds the AI model 160 by linking features between multiple layers so that a given input data set results in a specific output of the AI model. Thus, the modeler 120 can be configured to generate features and / or nodes of the AI model 160 as defined by the rules that link the features of the various layers. For example, a rule can link one or more features or nodes between an input and an output through the AI model (e.g., using relationship parameters including weights). That is, one or more linked features generate a rule. The AI model 160 is trained and includes a set of rules, each rule corresponding to a labeled or classified output. As described below, Figure 3B-2 A more detailed description of the functionality of modeler 120 is provided in .
[0096] In particular, if Figure 3B-2 As shown in, in the modeler 120, the identification features identified and / or extracted from the input data by the feature recognition engine 130 can be delivered to the classifier 360, which is configured to learn rules defining the relationship between features and outputs (e.g., predictions, actions, etc.). These features can be defined in one or more nodes located at one or more hierarchical levels of the trained AI model. Each feature can be linked to one or more features of other layers, where one or more relationship parameters define the interconnection between the first feature and other features (e.g., the second feature, the third feature, etc.) of other layers in the AI model.
[0097] For example, Figure 3B-2 As shown in , the classifier 360 is configured to determine which label or output a set of features (a group of generated rules) belongs to. That is, the rule links a given set of features to a specific output marked by the classifier 360, and the given set of features can be defined as a node within the AI model (i.e., a node of a training instance or point in the game process that describes the context of the game application). For example, a rule can link one or more features or nodes (links or relationships between features defined by one or more relationship parameters) through the AI model between the input and the output. The classifier can be configured to generate features and / or nodes of the AI model 160, wherein the features and / or nodes are used to define rules, as described above. The output can be associated with a label generated, assigned and / or determined by the classifier 360.
[0098] More specifically, for a given set of inputs and / or input data related to the game play process of the context of the game application, the learned rules can be learned paths and / or learned patterns (e.g., nodes or features leading to output nodes through the AI model). For example, one or more linked features and / or nodes generate rules. The trained AI model is a set of rules and labels (i.e., output). In a supervised learning environment, for a given set of features, the output is predetermined, and the deep learning engine learns the rules that link the set of features (e.g., by labels) to the output. In an unsupervised learning environment, a given set of features is not automatically bound to the output, where rules can be learned by finding similarities or clusters of data points associated with other similar feature groups. According to the success criteria defined for training, clusters may be superior to other clusters. In either case, existing rules for the input feature group can be matched, or new rules can be generated for the input feature group (which is likely to be similar to the evolution of one or more existing rules). The resulting output of the learned rules of the AI model can predict how successful the corresponding input feature group can be when used in the context of the game application. In addition, the output from the AI model (and optionally the input feature set) can be used (e.g., via an analyzer) to determine a course of action to be taken at a particular point in the game play process of the scenario (as determined by the condition of the game application or the game state indicated by the input data set). For example, the action may include a set of controller inputs suggested as recommendations, control commands that cause the game application to respond to the input, etc. One or more rules can be generated for a given set of features or a similar set of features. Based on the corresponding success criteria or reward criteria, a rule may be preferred for a given set of features or a similar set of features. For a given set of features, the most successful output can be selected for the given set of features. That is, for a given set of features (e.g., inputs), the most successful rule (and output) is selected for use in the trained AI model 160, which is evidence of incremental learning.
[0099] In a simple illustration, the success criterion may be defined by the number of points earned, where the path (e.g., link label) that generates the most points is more successful. In another example, the success criterion may be related to the level of the life bar of the player character or opponent, such as Figure 3C-1Other success criteria may be defined based on the player. For example, experts may be identified by one or more success criteria, and their corresponding game play may be used as training state data and generally used to learn the best actions to take for a given state or condition of the game play for the context of the game application. In this manner, success criteria may be defined to determine the skill level of the player, including how fast the player's response time is, how accurate the player is in aiming at one or more targets (e.g., typically, skilled players have a fast trigger and move quickly, decisively, and accurately from one target to another), how fast the cycle between controller inputs is, etc.
[0100] like Figure 3B-1 and Figure 3B-2 As shown in , the modeler 120 builds and / or outputs a trained AI model 160, which links the learned path and / or learned pattern to (e.g., links the label of the AI model to) a given set of inputs and / or input data related to the game play process of the context of the game application. The AI model 160 can be used later to provide one or more functionalities related to the game application and / or the game play process of the game application. That is, given a set of inputs that can indicate the conditions of the subsequent game play process of the player, the resulting output of the trained AI model 160 can be used (e.g., via an analyzer) to predict and / or determine the best action course to be taken at a specific point in the game play process of the context defined by the corresponding set of input data. For example, after the AI model 160 has been trained, the player may be playing the context of the game application. The player also encounters difficulties when advancing in the context, which may be reflected in the output of the AI model. New and subsequent input state data (e.g., game state) can be related to any data about that specific point in the game play process of the player (where difficulties are experienced). The input state data of the scenario is received and provided to the AI model via the deep learning engine 190, wherein the AI model can predict as an output how successful the game play process is when playing the scenario under given conditions of a given game application. The output from the AI model can be analyzed and used to perform various functionalities related to the game application and / or the game play process of the game application. For example, the output can be analyzed to determine the best course of action to be taken at the specific point in the game play process for the scenario. Actions can be performed based on the output. For example, the trained AI model 160 can provide recommendations to the player to advance his or her game play process.
[0101] For a given set of input data, other functionality may also be determined and generated based on the output of the AI model 160, where the functionality is in the form of output or action. That is, input state data is received during the first gameplay process of the scenario and provided as input to the deep learning engine. The deep learning engine applies the AI model to the input state data. An action is performed based on an analysis of the output of the AI model, where the action is performed in response to the input state data. FIG. 4A to FIG. 4B The application of an AI model for a given game play of a gaming application is shown. FIG. 5A to FIG. 5F Various illustrations of different actions or responses that can be performed according to predefined goals are provided. In particular, the AI model implemented by the deep learning engine 190 matches a given input state data with one or more rules defined within the trained AI model (each rule provides linked or interconnected nodes and / or features). Each rule is associated with an output. Success criteria can be applied to generate rules. In addition, the analyzer 140 obtains the output and performs additional analysis to determine the appropriate action associated with the corresponding input data. For example, when a rule is satisfied relative to a success criterion for a given set of input state data, a corresponding action can be identified and / or executed.
[0102] Figure 3C-1 It is a screenshot 300C of the game playing process of a game application according to an embodiment of the present disclosure, and shows a type of success criterion that can be used to train an AI model that understands how to play the game application. In particular, player P-1 controls the game playing process. As shown in the figure, screenshot 300C shows a front view of the battle between Kratos 391 and enemy 396. In the "God of War" game application, Kratos is a Spartan warrior in Greek mythology, and his mission is to kill the god of war Ares. During the game playing process, the player can control Kratos 391. The training state data 320a from the game playing process associated with screenshot 300C is fed to the depth and / or machine learning engine 190 for the purpose of training the AI model, as previously described. In addition, the success criterion 330 is supplied to the depth and / or machine learning engine 190, wherein the deep learning engine 190 uses the success criterion to train the AI model. For example, the success criterion can be used to distinguish the path through the AI model (wherein the path crosses the node in the AI model). In an embodiment, one or more success criteria may be defined for training the AI model. For example, the success criteria may include earning the most points, earning the most assets, or the most powerful assets or key assets, etc.
[0103] In one example, success criteria 330 can be defined by whether the life bar 397 of the enemy 396 during the game process becomes low, wherein the life bar indicates the health of the corresponding character. In addition, the life bar 392 indicates the health of Kratos 391 during the game process. As shown, the life bar 397 of the enemy is extremely low, and it can be indicated that the game process is successful or has a high degree of success. In another example, the success criteria may be more complex and defined by the speed at which the life bar 397 of the enemy 396 is exhausted. If the life bar 397 is exhausted quickly, this indicates that the game process is being controlled by a highly skilled player. For example, a highly skilled player understands how to fight the enemy 397, and knows the controller input sequence for defeating the enemy 396, and has the skills and / or ability to quickly and accurately (e.g., without bias) implement those sequences. During the training of the AI model, the success criteria using the speed of focusing on the life bar exhaustion can distinguish between successful input and less successful input, and help identify the controller input sequence that will be successful if implemented by another player during the specific conditions or game state of the game process.
[0104] Figure 3C-2 It is a diagram of collecting training state data from multiple game playing processes according to an embodiment of the present disclosure, wherein multiple instances are being executed on multiple servers to support the game playing process. A plurality of game playing processes 310 (e.g., 310a to 310n) of a game application are shown. The plurality of game playing processes 310 are controlled by a plurality of players P-1 to pn through corresponding client devices. As shown, the game playing process provides various game playing process data 320a to 320n, wherein the extracted data can be used as training state data 345. The game playing process data may include metadata, including game state data, as previously described. Each game playing process is in the context 1 (S-1) of the game playing application, wherein each game playing process is unique and has a unique result. That is, each game playing process can be associated with a measured degree of success of advancing through the context S-1. The degree of success can be measured in part using a success criterion 330, wherein the success criterion 330 can be used by the modeler 120 of the depth and / or machine learning engine 190 to train the AI model 160, as previously described. For example, the success criterion can be used to distinguish similar training state data for the purpose of training the AI model.
[0105] According to one embodiment of the present disclosure, through a detailed description of various modules of system 100A and system 200B, flowchart 400A discloses a method for applying an AI model, wherein the AI model knows everything about a game application and / or a game process of the game application based on one or more success criteria (e.g., understands how to play the game application), and the AI model is used to provide various functionalities related to the game application and / or the game process of the game application. As previously described, flowchart 400A can be implemented in a backend server (e.g., implemented in combination with deep learning engine 190 in game cloud system 201).
[0106] In particular, at 410, the method includes training an AI model from a plurality of game play sessions of a context of a gaming application. For example, the AI model is trained to play the context using training state data collected from a plurality of game play sessions of the context of the gaming application and associated success criteria for each game play session. That is, the AI model is trained to learn the complexity of the gaming application and / or the complexity of playing the gaming application or the context of the gaming application.
[0107] In one embodiment, the AI model is trained using training state data collected during multiple game play processes of a game application, wherein multiple game play processes are controlled by multiple players via multiple client devices. For example, multiple servers may be executing multiple instances of a game application, wherein these instances support multiple game play processes. Training state data is collected at multiple servers, wherein the training state data is associated with the corresponding game play process of the situation. For example, the training state data may include metadata associated with the game play process, including controller input, game states that define the state and / or conditions of the game application at a specific point during the game play process, progress in the game play process, the result of the situation (e.g., success or failure), user profile information, etc., as previously described. That is, the training state data includes any data that may be relevant to understanding the game application and / or the game play process of the game application.
[0108] In addition, one or more success criteria are defined for the training state data. For example, the success criteria are used to train an AI model that understands how to play a game application. For example, the success criteria can be used to define favorable links between nodes of the AI model, or to define favorable paths through the nodes of the AI model for a given set of input data. The success criteria and the training state data are provided to the deep learning engine to train the AI model.
[0109] At 420, the method includes receiving first input state data during a first game play process of the scenario. In one embodiment, the first game play process occurs after the AI model is trained. That is, the first game play process is not used to train the AI model. In another embodiment, the first game play process may occur during training, so that the first game play process is used to train the AI model (e.g., self-training mode). In another embodiment, the first game play process may not be used to train the AI model initially, and is mainly used to determine the appropriate action to be taken given the first input state data. Thereafter, the first game play process (e.g., the first input state data) can be used to refine the AI model through additional training.
[0110] At 430, the method includes applying the first input state data to the AI model to generate an output. In one embodiment, the output may indicate a degree of success of the scenario for the first game play. That is, the output may predict how successful the first game play will be when progressing through the scenario.
[0111] At 440, the method includes performing an analysis of the output based on a predefined goal. In addition, the group of inputs (e.g., current and past groups of inputs) may also be analyzed. Based on the predefined goals, the analysis may generate actions to be performed at specific points in the corresponding game play process for the context (as determined by the condition of the game application or the game state indicated by the input data set). For example, if the predefined goal is to provide assistance, the analysis may generate recommendations or suggestions on how to advance through the conditions encountered during the game play process of the context of the game application. If the predefined goal is to provide guidance, the analysis may determine the player's weaknesses and provide the player with a tutorial session to address the weaknesses. Support other predefined goals, such as providing game support, providing peers during game play, automatically training AI models, discovering defects in game applications, automatically testing game applications, etc.
[0112] Thus, at 450, the method includes performing actions based on the analyzed output to achieve a predefined goal. FIG. 5A to FIG. 5F Various illustrations and examples are provided of different actions or responses that can be performed based on predefined goals.
[0113] Figure 4B A method for applying an AI model configured to understand how to play a game application according to one embodiment of the present disclosure Figure 2B1 is a block diagram of an AI processor engine 210 for providing various functionalities related to a game application and / or a game play process of the game application. As shown, the AI processor engine 210 includes a deep learning engine or machine learning engine 190 configured to train and / or apply an AI model 160, wherein the AI model 160 is configured to provide an output for a given input. The AI processor engine 210 also includes an analyzer for using the output from the AI model 160 to determine a response to the input.
[0114] In particular, input state data 405 is provided as input to a deep learning engine 190, which is configured to implement an AI model 160. The input state data may be associated with the game play process of the game application, including controller input, game state data, user data, success criteria, etc., as previously described. In some embodiments, the input state data 405 may be provided after training the AI model 160. In other embodiments, the input state data 405 may be provided during the training (e.g., self-training AI model) of the AI model 160. As previously described, the AI model 160 generates output 173 for a given input state data 405. For example, the output may indicate how successful the game play process may be in advancing through the context of the game application. The output may be analyzed by an analyzer 140 to provide various functionalities related to the game play process of the game application. The analyzer 140 determines appropriate actions or responses according to predefined goals, as previously described (e.g., guidance, providing assistance, etc.). That is, the analyzer 140 may use the AI model 160 to determine what is needed to guide the game play process to a successful end during the context. Thus, the analyzer 140 can determine and execute actions according to predefined goals for a given input state data. In this way, the analyzer can provide various functionalities related to the game application and / or the game play process of the game application by using the AI model 160.
[0115] As shown, as part of the analysis, the analyzer 140 can perform one or more functions. For example, the player analyzer 144a of the analyzer 140 is configured to perform the analysis of the player playing the game application (for example, determining the skill level of the player). The weakness identifier 141a of the analyzer 140 is configured to identify the weaknesses in the game ability of the player. The input control sequence parser 147a of the analyzer 140 is configured to determine the controller input sequence used by the player to control the game process. The sequence diagram and / or route tracker 148a of the analyzer 140 are configured to track the progress of the game process, including tracking the progress in the game environment. For example, the tracker 148a is configured to track the route taken in the game environment, or build a map of the game environment, or build a node diagram of the game process indicating the progress of the game application. Other modules that provide analysis of the output of the AI model for a given input state data 405 are also supported.
[0116] The analyzer 140 is also configured to determine and execute an action by the action generator 170 in response to the analysis of the output determined in response to the given input state data 405. The action is determined and executed according to the predefined goal. For example, the action can provide services to players playing the game application (e.g., providing a profile of the player playing the game application; providing recommendations to the player during the game process of the player playing the game application, wherein the recommendations can be structured taking into account the user profile; discovering the player's weaknesses and providing services to address these weaknesses; training the player; providing a robot opponent to the player; taking over the player's game process, etc.). The action can also provide services to game developers or other entities, such as discovering defects in the game application, using player profiles that compete with each other in the game application (such as a multiplayer online game) to build an equal team, automatically training AI models through automatic play, exploring the game environment of the game application to discover defects, etc. FIG. 5A to FIG. 5F Provides various illustrations of different actions or responses that can be performed based on predefined goals.
[0117] In particular, Figure 5AA data flow diagram according to one embodiment of the present disclosure is shown, which illustrates a process for providing recommendations to players playing a game application, the recommendations being provided by an AI model 160 trained by a backend server network executing an instance of the game application. A game play process 501x of the game application is shown. As previously described, the game play process can be controlled by a player Px through a corresponding client device, where an instance of the game application is executed at a backend server of a streaming game service (e.g., a cloud gaming system). In other embodiments, the game application can be executed locally on the client device, and metadata is delivered to the backend server for AI model support. The game play process 501x occurs during scenario S-1.
[0118] Input state data 505x from the game play process 501x is provided to an analyzer 140, which is configured to analyze the output of the trained AI model 160, where the AI model 160 is implemented by the deep learning engine 190 (in the application phase rather than the learning phase). The input state data 505x is received after the AI model 160 is trained, and therefore may not be part of the training state data used to train the AI model, as previously described. The AI model 160 is configured to provide an output 173, and the analyzer 140 is configured to perform or provide an action to be performed based on the analysis of the output 173.
[0119] In particular, the action generator 170 of the analyzer 140 includes a recommendation engine 145a, which is configured to provide recommendations to players during the game process of the player along the game application. For example, a predefined goal can be to provide game support to the player Px, as implemented by the analyzer 140. Recommendations can be provided in response to conditions (e.g., game state) during the game process, such as when the player has difficulty in advancing in a specific context of the game application. In one embodiment, the analyzer 140 can determine that the player is struggling by determining that the features in the first input state data or the first input state data do not meet the corresponding success criteria. The success criteria can provide an indication of how successful the game process will be, and how successful the described part of the game process will be in particular. For example, the success criteria can indicate how many scores have been accumulated during the context. When the analyzer 140 determines that the first input state data does not meet the success criteria from the output of the AI model 160, then a recommendation 515 can be provided as an action to the client device of the player Px.
[0120] In one embodiment, the recommendation may take into account the user / player profile 510x of player Px (e.g., taking into account the player's skill level), wherein the player profiler 144a may analyze the output data and / or input state data from the trained AI model 160 to determine the user profile 510x of the situation (e.g., how the player reacts to the game application during the situation, skill level, and other player characteristic metrics). That is, the recommendation reflects the skill of player Px. In other embodiments, the user profile 510x may be determined by the historical game play process of the subject game application and other game applications. For example, if the user profile 510x indicates that the player is an expert player, then the recommendation 515 for player Px may provide a small hint without too much detail. On the other hand, if the user profile 510x indicates that player Px is a novice player, then the recommendation 515 may provide detailed instructions for the player to follow.
[0121] In one embodiment, the recommendation 515 may include a query asking the player Px whether he wants to participate in the auto-play feature. If so, the task auto-play engine 145b is configured to take over the game play process of the player Px. In this way, the task auto-play engine 145b automatically (and successfully) performs the situation or tasks within the situation in the game application.
[0122] Figure 5B A data flow diagram according to an embodiment of the present disclosure is shown, and the data flow diagram shows a process of balancing a player team of a game application using a player profile determined by an AI model trained by a back-end server network of an instance of the game application. A plurality of game play processes 501 (e.g., 501a to 501n) of the game application are shown. The game play process can be controlled by a plurality of players P-1 to pn through corresponding client devices. As previously described, an instance of the game application is executed on a back-end server of a streaming game service (e.g., a cloud game system), as previously described. In other embodiments, the game application can be executed locally on a client device, and metadata is delivered to a back-end server for AI model support. For example, the game play process 501a to 501n can occur during one or more scenarios S-1, S-2, and S-3.
[0123] Input state data 505a to 505n from the game play process 501a to 501n are provided to the analyzer 140, which is configured to analyze the output of the trained AI model 160, wherein the AI model 160 is implemented by the deep learning engine 190 (in the application phase rather than the learning phase). In one embodiment, a plurality of input state data 505a to 505n are received during the second plurality of game play processes of the context of the game application, wherein the game play process can be performed after the training AI model. The second plurality of game play processes are controlled by a plurality of players, and wherein the plurality of input state data include a plurality of player characteristic metrics of a plurality of players. The plurality of input state data are applied to the AI model to generate a plurality of outputs indicating a plurality of success degrees of the second plurality of game play processes of the context. In addition, based on a plurality of player characteristic metrics, a plurality of outputs are analyzed to determine a plurality of player profiles. The AI model 160 is configured to provide an output, and the analyzer 140 is configured to perform or provide an action to be performed based on the analysis of the output. In particular, the analyzer 140 is configured to build a balanced player team from a plurality of players based on the corresponding player profiles to achieve a predefined goal.
[0124] During the analysis, the analyzer 140 may perform user profiling, such as by a player profiler 144a. In particular, the output from the trained AI model 160 is provided to the player profiler 144a, which is configured to provide a profile of a player P-1 to Pn playing the game application. The player profiles 510a to 510n may be limited to the game play process of a specific situation of the game application (e.g., the profile determines or predicts how the corresponding player reacts to the game application during the situation, the skill level of the corresponding player, etc.). For example, the player profile may be defined based on a plurality of player characteristic metrics, wherein each player profile includes one or more corresponding player characteristic metrics. Input state data may be received during the game play process of the player playing the game application. The input state data includes a plurality of player characteristic metrics of a plurality of players. As previously described, the input state data is also applied to the AI model 160 to determine the output. The player profiler 144a analyzes the output 173 and / or the input state data 505a to 505n to determine the corresponding player profile of the corresponding player. For example, the corresponding player profile is based on the corresponding player characteristic metrics provided as input to the trained AI model. For example, the player characteristic metrics may include: the accuracy of the player; or the speed at which the corresponding player generates input control data sequences; or the reaction time of the corresponding player in responding to events in the context; or the consistency of the corresponding player; or the transition time of the corresponding player between the first goal and the second goal, etc. In other embodiments, the corresponding player profile can be determined through historical game play of the subject game application and other game applications, as previously described.
[0125] In addition, the action generator 170 of the analyzer 140 includes a team balancer module 144b, which is configured to use the player profile to build a fairly balanced team (e.g., a team competing with each other in a game application) according to a predefined goal. For example, the predefined goal can be to provide parity during the game process (e.g., in a massively multiplayer online game application-MMO), as implemented by the analyzer 140. For example, based on the player profiles 510a to 510n, the team balancer 144b is configured to build a fairly balanced team (e.g., a team competing with each other in a game application). In this way, the game process between the two teams of the game application will be fair, attractive, and valuable to the players of each team. For example, a balanced team will help avoid a suspenseful match. Consider a player profile that is further divided into two groups (G1 and G2) based on the skill level of one or more categories. It is expected that the players are dispersed, and / or the skills of each group of players are balanced in all teams (e.g., team 1 to team n). Each team in a battle-centric game application includes one or more positions, including a leader, a sniper, and a trained killer (e.g., hand-to-hand combat). In one description, you can group all skill positions and then distribute them evenly between teams. In the description, the leader is excluded, and for simplicity, all leaders are selected from a group G-2 so that all teams have equally strong leaders. Assume that all other skill positions are equally important. In other words, each team should be composed of many skill positions, and the same number of players are selected from each group G1 and G2. For example, each team includes a player from group G1, and a player from group G2 (excluding the leader). As shown in the figure, team 1 and team 2 each include a player from group G1 (sniper) and a player from G2 (e.g., killer). In addition, team 3 includes a player from group G1 (killer) and a player from G2 (e.g., sniper). Thus, each team is balanced according to user profiles (e.g., skill level).
[0126] Figure 5CA data flow diagram according to one embodiment of the present disclosure is shown, which shows the process of training an AI model through automatic play guided by an AI model 160. As previously described, the AI model 160 is trained by a back-end server network that executes an instance of a game application. A plurality of game play processes 310a to 310n of the game application are shown. As previously described, game play process data 320a to 320n are generated from the game play processes 310a to 310n. In one embodiment, during training, a plurality of game play processes are controlled by the AI model 160. In this way, the analyzer 140 uses data from the AI model 160 to generate input state data, wherein the input state data is provided by the analyzer 140 as a previous action during training. The game play process data may include metadata, including game state data describing the game state at a specific point, and may include controller input data, and may include a record of the game play processes 310a to 310n in order to extract metadata and / or training state data. The capture engine 340 captures the game play data 320a to 320n, as well as other data that may be provided, such as success criteria 330, as previously described. The success criteria can be used to distinguish similar training state data for training purposes. That is, the success criteria can be used by the deep learning engine 190 to train the AI model 160. The training state data 345 is provided to the deep learning engine 190. The functions of the deep learning engine 190 are related to FIG. 3B1 and Figure 3B-2 described, and also applies to Figure 5C The deep learning engine 190 shown. For simplicity, Figure 5C All components of the deep learning engine 190 are not shown in the figure. Typically, the deep learning engine 190 includes a modeler 120, which is configured to train and / or build an AI model 160 using training state data 345 and success criteria 330. The modeler 120 can implement artificial intelligence through various neural networks (e.g., convolution, recursion, etc.). In particular, the modeler 120 identifies a set of feature-related rules that predict and / or determine the actions to be taken given a set of inputs (e.g., features that can define the scenario or condition of the game application-the game state). For example, the output of the AI model 160 can predict the success of a given set of input data (e.g., defining the conditions of the game process of the game application) in the context of the game application. A set of rules connecting features and / or nodes constitutes the AI model 160.
[0127] As shown, the deep learning engine works in both the learning and application stages. In particular, the game playing process 310 is automatically executed such as by the AI model 160. In this way, the AI model 160 is self-training. Specifically, the analyzer 140 includes an input control sequence parser 571, a player profiler 144a and a map / route tracker 572, each of which has been previously introduced. For example, the input control sequence parser 147a is configured to determine the controller input sequence that the player uses to control the game playing process. The sequence diagram and / or route tracker 148a of the analyzer 140 are configured to track the progress of the game playing process, including tracking the progress in the game environment. The player profiler 144a of the analyzer 140 is configured to perform the analysis of the player of the game application (for example, determining the skill level of the player).
[0128] In addition, the action generator 170 of the analyzer 140 includes an automatic player 143a, which is configured to automatically play the game application under the guidance of the AI model trainer 143b according to a predefined goal, such as for the purpose of automatically training the AI model. For example, the predefined goal can be to automatically train the AI model 160. Specifically, the analyzer 140 includes an automatic player 143a, which is configured to automatically play the game application under the guidance of the AI model trainer 143b. For example, for a given set of training state data, such as for a training instance, the analyzer 140 can analyze the learned output of the AI model 160 to determine the next set of controller inputs for controlling the corresponding game process. In particular, the robot selector and input control selection / prediction engine 537 of the AI model trainer 143b combination are configured to determine the next set of controller inputs according to the goal. For example, a robot that is optimized for completing the game with the best success and efficiency or exploring different arrangements of the game application can be selected. According to the goal, the input control selection / prediction engine 537 is configured to predict what the next set of controller inputs are in the case of a given goal and a selected robot. The controller input is delivered back to the automated player 143a, which then sends the controller input to the corresponding instance of the game application supporting the corresponding game play process. In this way, by utilizing a set of back-end servers to execute instances of the game application for the corresponding game play processes 310a to 310n, training state data 345 can be automatically generated quickly and efficiently.
[0129] In one embodiment, for a first input state data, multiple groups of input controls or controller inputs to be performed after the first input state data can be determined. A first group of input controls is selected, which is predicted to meet the success criteria and is predicted to have the highest degree of success among the multiple groups of input controls. In this way, the first group of input controls can be delivered as an action to the corresponding instance of the game application for execution.
[0130] Figure 5D A data flow diagram according to an embodiment of the present disclosure is shown, which illustrates a process of automatically sweeping a game application using an automatic sweeper / player 146a under the guidance of an AI model 160, which is trained by a back-end server network executing an instance of a game application. A plurality of game play processes 501 (e.g., 501a to 501n) of a game application are shown. The game play process can be automatically controlled such as by an AI model 160. In this case, the AI model 160 can be configured to provide the next input state data (e.g., a controller input to generate a game state and other input state data). An instance of a game application can be executed on a back-end server of a streaming game service (e.g., a cloud gaming system). For example, a game play process 501a to 501n can occur during one or more situations S-1, S-2, and S-3. Input state data 505a to 505n from the game play process 501a to 501n is provided to an analyzer 140, which is configured to analyze the output of the AI model 160. The deep learning engine 190 implements the AI model 160 (in the application phase rather than in the learning phase). The AI model 160 is configured to provide an output, and the analyzer 140 is configured to perform or provide an action to be performed based on an analysis of the output.
[0131] In particular, the game play process 310 is automatically executed such as by the AI model 160. For example, after the AI model has been trained, the predefined target can automatically test the game application. For example, the AI model may have previously automatically trained itself, and the AI model can now be used to test the game application. In another embodiment, the automatic sweep feature is performed during training. Specifically, the analyzer 140 includes an input control sequence parser 571 and a map / route tracker 572, each of which has been previously introduced. For example, the input control sequence parser 147a is configured to determine a controller input sequence previously tested. The sequence diagram and / or route tracker 148a of the analyzer 140 are configured to track the progress of the game play process during testing, including tracking the progress in the game environment.
[0132] In one embodiment, the first input state data is generated by the AI model as an action previously taken during training. During analysis of the output of the AI model, different permutations are then determined to respond to the first input state data. For example, each permutation includes a unique set of controller inputs to be acquired. Thereafter, one or more actions are taken, such as executing a different permutation. In this way, the system can be configured to discover any anomalies when playing a game application.
[0133] In particular, the action generator 170 of analyzer 140 comprises automatic sweeper engine / player 146a, which is configured to explore game applications such as by using arrangement engine 146b to determine the various arrangements that game applications can perform. For example, input control arrangement engine 146b is configured to determine the various arrangements that game applications can perform such as in response to given conditions (for example, specific game state). That is to say, for given input state data 405a to 405n (for example, game state), input control arrangement engine 146b can determine what the next group of controller input should be. In this case, arrangement engine 146b finds different arrangements for responding to input state data, and each arrangement comprises a group of different actions to be taken. Automatic sweeper engine 146a is then configured to experience various arrangements by controlling playing game process 401a to 401n (for example, by submitting suitable controller input to the execution instance of game applications).
[0134] In addition, the analyzer 140 is configured to perform quality analysis on the game application, such as to find weaknesses (e.g., overly long and boring sequences, difficult sections, etc.) or flaws (e.g., glitches, loops, etc.) in the game application. For example, the map / route analyzer 441 is configured to analyze the outputs (e.g., game states) of different arrangements of the game application to find weaknesses in the game application. In one implementation, the game code identifier 443 is configured to find problems in the coding of the game application, wherein the code location 447 is provided as an output.
[0135] Figure 5E A data flow diagram according to one embodiment of the present disclosure is shown, which illustrates a process of providing an opponent for a player, wherein the opponent is guided by an AI model trained by a back-end server network executing an instance of a game application. A game play process 501x of a game application is shown. As previously described, the game play process can be controlled by a player Px through a corresponding client device, wherein an instance of the game application is executed at a back-end server of a streaming game service (e.g., a cloud gaming system). In other embodiments, the game application can be executed locally on a client device, and metadata is delivered to a back-end server for AI model support. The game play process 501x occurs during context S-1 and is controlled by a player input control 503x.
[0136] Input state data 505x from the game play process 501x is provided to an analyzer 140, which is configured to analyze the output of the trained AI model 160, where the trained AI model 160 is implemented by the deep learning engine 190 (in the application phase rather than the learning phase). The input state data 505x is received after the AI model 160 is trained, and therefore may not be part of the training state data used to train the AI model, as previously described. The AI model 160 is configured to provide an output, and the analyzer 140 is configured to perform or provide an action to be performed based on the analysis of the output.
[0137] Specifically, the analyzer 140 includes an input control sequence parser 571 and a player profiler 144a, each of which has been previously introduced. For example, the input control sequence parser 147a is configured to determine the controller input sequence used by the player to control the game play process. The player profiler 144a of the analyzer 140 is configured to perform profiling of the player playing the game application (e.g., determining the skill level of the player).
[0138] In addition, the analyzer 140 includes a robot builder 575, which is configured to build one or more robots (automatic player robots or opponents for playing games or for controlling characters in the game process). For example, the robot builder 575 can be configured to build an ultimate robot 142d, which is learned by the AI model 160 by applying success criteria. As previously described, for a given set of inputs (e.g., input training data), more successful patterns (e.g., rules including linking features and / or tags) are learned and selected rather than less successful patterns. In this way, the best robot or ultimate robot 142d that is most successful when playing the game application is trained. In another example, the robot builder 575 is configured to build a virtual player or a virtual me robot 142c that simulates the first player. In one embodiment, the training state data is obtained from the first player playing the game on one or more client devices. That is, data from other players' game play processes are not used to train the virtual me robot 142c. In this way, the AI model is learned from the metadata generated by the first player's game play process, and therefore, the AI model will be a direct reflection of the first player.
[0139] In another example, the robot builder 575 is configured to build one or more robots of different skill levels. For example, robots of different skills may include expert robots 576, medium robots 577, and beginner robots 578. In one implementation, the training state data may be parsed to reflect the corresponding skill level so that the AI model is trained using data of the corresponding skill level. For example, a success criterion may be defined so that only the game play of an expert player is used to train the AI model so that the robot builder 575 may build an expert robot 576. In another example, a success criterion may be defined so that only the game play of a medium-skilled player is used to train the AI model so that the robot builder 575 may build a medium robot 577. In yet another example, a success criterion may be defined so that only the game play of a beginner-skilled player is used to train the AI model so that the robot builder 575 may build a beginner robot 578. In yet another embodiment, a robot of a specific skill level may be implemented by using a virtual ultimate robot 142d and applying one or more conditions to the performance of the ultimate robot, including introducing randomness and / or waiting time. For example, the performance of the ultimate robot can be compromised by introducing waiting times between the execution of controller inputs in a sequence, or by introducing random controller inputs to a given sequence of controller inputs that is known to be highly successful in completing a task.
[0140] In addition, the action generator 170 of the analyzer 140 includes an automatic player 143a, which is configured to automatically play the game application under the guidance of the AI model trainer 143b according to a predefined goal, such as for the purpose of automatically training the AI model. For example, the predefined goal can be to automatically train the AI model 160. Specifically, the analyzer 140 includes an automatic player 143a, which is configured to automatically play the game application under the guidance of the AI model trainer 143b.
[0141] The analyzer 140 includes an automatic player 143a, which is configured to automatically play the game application for the purpose of providing a robot opponent (e.g., an automatic reaction robot opponent). The robot opponent selector 142a is configured to select an appropriate robot, such as those robots previously introduced (e.g., ultimate robot 142d, virtual me robot 142c, and robots of different skills, including expert robot 576, medium robot 577, or beginner robot 578). The player who controls the game process can control the character that confronts the robot opponent. In particular, the automatic player 143a is configured to implement the selected automatic robot (robot). For a given set of input state data 505x, the automatic player 143a can be output by the analyzer 140 to determine the next set of instructions for controlling the robot found in the game application.
[0142] In addition, a robot suppression mechanism (bot throttler) 142b is configured to apply a difficulty setting reflected in the corresponding robot (e.g., applied to the operation of the robot). The robot suppression mechanism can start from the ultimate robot 142d or any other learned robot. The difficulty setting can be actively applied by the player, or it can be applied according to the user profile. For example, if the player is an expert indicated by the profile, the difficulty will be set higher (the opponent robot is difficult to defeat). On the other hand, if the player is a beginner, the difficulty setting will be lower (the opponent robot is easy to defeat). As shown, the robot suppression mechanism 142b may include a randomizer 551 configured to introduce random instructions. A corresponding robot with a high difficulty will have a low level of random instructions inserted into the instruction stream that is usually used to control the ultimate robot 142d. Thus, the resulting robot will be similar to the ultimate robot 142d. On the other hand, a resulting robot with a low difficulty will have a high level of random instructions inserted into the instruction stream that is usually used to control the ultimate robot 142d. Because there are random instructions, the resulting robot will act unstable and perform worse than the ultimate robot 142d. In addition, a waiting time engine 553 can be implemented to apply the difficulty setting. For a resulting robot with a high difficulty, the waiting time introduced in the instruction stream that is typically applied to the ultimate robot 142d will be limited or no waiting time. On the other hand, for a resulting robot with a low difficulty (e.g., for a beginner), the amount of waiting time introduced in the instruction stream that is typically applied to the ultimate robot 142d will be large. In this way, a resulting robot with a low difficulty setting will then move very slowly, such as during an attack, or during a defensive action, and thus may be easily defeated.
[0143] Fig. 5F A data flow diagram according to one embodiment of the present disclosure is shown, which illustrates a process for identifying a player's weaknesses and providing various services to train the player to overcome those weaknesses. A game play process 501x of a game application is shown. As previously described, the game play process can be controlled by the player Px through a corresponding client device, where an instance of the game application is executed at a backend server of a streaming game service (e.g., a cloud gaming system). In other embodiments, the game application can be executed locally on the client device, and metadata is delivered to the backend server for AI model support. The game play process 501x occurs during scenario S-1 and is controlled by player input control 503x.
[0144] Input state data 505x from the game play process 501x is provided to an analyzer 140, which is configured to analyze the output of the trained AI model 160, where the trained AI model 160 is implemented by the deep learning engine 190 (in the application phase rather than the learning phase). The input state data 505x is received after the AI model 160 is trained, and therefore may not be part of the training state data used to train the AI model, as previously described. The AI model 160 is configured to provide an output, and the analyzer 140 is configured to perform or provide an action to be performed based on the analysis of the output.
[0145] In addition, the action generator 170 of the analyzer 140 includes an automatic player 143a, which is configured to automatically play the game application under the guidance of the AI model trainer 143b according to a predefined goal, such as for the purpose of automatically training the AI model. For example, the predefined goal can be automatically training the AI model 160.
[0146] In particular, the analyzer 140 includes a weakness identifier 141a configured to determine a weakness of a corresponding player controlling a corresponding game play process according to a predefined goal to provide guidance. The weakness is determined by analyzing the game play process of the player.
[0147] For example, the analysis may include a comparison with a success criterion. In particular, a weakness may be identified by determining that the first input state data has a lower than average degree of success, as determined by an analysis of a comparison of an output from the AI model with other outputs generated from comparable input state data. As an illustration, for a given input state data (e.g., a first input state data), a first set of interconnected nodes that generate an output in response to the first input state data may have a lower value than the average value determined for a similar game play process with comparable input state data when the corresponding success criterion is met. That is, the comparison is performed with a game play process with the same or similar input state data. Thus, the weakness identifier 141a can determine the player's weakness.
[0148] In addition, the weakness trainer 141b is configured to perform services that help players overcome weaknesses. For example, the weakness trainer 141b can provide one or more tutorials 561 (e.g., videos, game sessions, etc.) intended to improve the player's skills related to the player's weaknesses. The tutorial can include a video tutorial that provides instructions on how to strengthen the player's skills related to the weaknesses, or a game session that includes skills intended to strengthen the skills related to the weaknesses. In addition, a training session that solves the identified weaknesses can be presented to the player via a corresponding client device. In addition, in one embodiment, the weakness trainer 141b can be configured to provide a robot that is dedicated to guiding the game play process in a way that exposes the player's weaknesses. The robot can be a virtual ultimate opponent 565 that the player confronts. The virtual ultimate opponent 565 can be an ultimate robot 142d, as previously described.
[0149] Figure 6 Components of an exemplary apparatus 600 that can be used to perform aspects of various embodiments of the present disclosure are shown. For example, according to one embodiment of the present disclosure, Figure 6 An exemplary hardware system suitable for training an AI model capable of performing various functionalities associated with a game application and / or the game play of a game application is shown. The block diagram shows a device 600, which may be incorporated into or may be a personal computer, a server computer, a game console, a mobile device, or other digital device, each of which is suitable for practicing embodiments of the present invention. The device 600 includes a central processing unit (CPU) 602 for running software applications and optionally an operating system. The CPU 602 may be composed of one or more homogeneous or heterogeneous processing cores.
[0150] According to various embodiments, CPU 602 is one or more general-purpose microprocessors with one or more processing cores. Other embodiments can be implemented using one or more CPUs with a microprocessor architecture, which is particularly suitable for highly parallel and computationally intensive applications configured for deep learning, content classification and user classification, such as media and interactive entertainment applications. For example, CPU 602 can be configured to include an AI engine (e.g., deep learning) 190, which is configured to support and / or perform learning operations on providing various functionalities (e.g., predictions) related to the game playing process of a game application and / or a game application. The deep learning engine may include a modeler 120, which is configured to build and / or train an AI model, which is configured to provide various functionalities related to the game playing process of a game application and / or a game application. In addition, CPU 602 includes an analyzer 140 configured to implement a trained AI model. The trained AI model provides an output in response to an input, wherein the output depends on the predefined functionality of the trained AI model. The trained AI model can be used to determine what actions can be taken during the game playing process. The analyzer 140 determines which is the appropriate action to take. That is, the analyzer 140 is configured to perform various functionalities related to the game application and / or the game play process of the game application. The analyzer 140 is configured to analyze the output from the trained AI model 160 for a given input (e.g., controller input, game state data, success criteria) and provide a response.
[0151] Memory 604 stores applications and data for use by CPU 602. Storage device 606 provides non-volatile storage and other computer-readable media for applications and data and may include fixed disk drives, removable disk drives, flash memory devices and CD-ROM, DVD-ROM, Blu-ray disc, HD-DVD or other optical storage devices, as well as signal transmission and storage media. User input device 608 communicates user input from one or more users to device 600, examples of which may include keyboard, mouse, joystick, touch pad, touch screen, static or video recorder / camera and / or microphone. Network interface 614 allows device 600 to communicate with other computer systems via an electronic communication network, and may include wired or wireless communications on a local area network and a wide area network such as the Internet. Audio processor 612 is adapted to generate analog or digital audio output from instructions and / or data provided by CPU 602, memory 604 and / or storage device 606. The components of device 600 , including CPU 602 , memory 604 , data storage device 606 , user input device 608 , network interface 610 , and audio processor 612 , are connected via one or more data buses 622 .
[0152] The graphics subsystem 614 is further connected to the data bus 622 and the components of the device 600. The graphics subsystem 614 includes a graphics processing unit (GPU) 616 and a graphics memory 618. The graphics memory 618 includes a display memory (e.g., a frame buffer) for storing pixel data for each pixel of an output image. The graphics memory 618 can be integrated in the same device as the GPU 616, connected to the GPU 616 as an independent device and / or implemented in the memory 604. The pixel data can be provided directly from the CPU 602 to the graphics memory 618. Alternatively, the CPU 602 provides the GPU 616 with data and / or instructions defining the desired output image, and the GPU 616 generates pixel data for one or more output images based on these data and / or instructions. The data and / or instructions defining the desired output image can be stored in the memory 604 and / or the graphics memory 618. In an embodiment, GPU 616 includes 3D rendering capabilities for generating pixel data for an output image based on instructions and data defining the geometry, lighting, shading, textures, motion, and / or camera parameters of a scene. GPU 616 may also include one or more programmable execution units capable of executing shader programs. In one embodiment, GPU 616 may be implemented within AI engine 190 to provide additional processing capabilities, such as for AI or deep learning functionality.
[0153] Graphics subsystem 614 periodically outputs pixel data of an image from graphics memory 618 for display on display device 610 or for projection by projection system 640. Display device 610 may be a device that is capable of responding to a video from device 600.
[0154] Any device that uses a signal to display visual information, including CRT, LCD, plasma, and OLED displays. For example, device 600 can provide analog or digital signals to display device 610.
[0155] Thus, in various embodiments, the present disclosure describes systems and methods for implementing deep learning (also known as machine learning) techniques to build AI models using training data collected from a network of servers executing instances of a gaming application that supports one or more gameplay processes, and using the trained AI models to provide various functionalities related to the gaming application and / or the gameplay process of the gaming application.
[0156] It should be understood that the various embodiments defined herein can be combined or assembled into specific implementations using the various features disclosed herein. Therefore, the examples provided are only some possible examples, and are not limited to various implementations that are possible by combining various elements to define more implementations. In some examples, some implementations may include fewer elements without departing from the spirit of the disclosed or equivalent implementations.
[0157] Embodiments of the present disclosure may be practiced with various computer system configurations including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. Embodiments of the present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices linked over wired or wireless networks.
[0158] In view of the above embodiments, it should be understood that many embodiments of the present disclosure may employ various computer-implemented operations involving data stored in a computer system. These operations are operations that require physical manipulation of physical quantities. Any operation described herein that forms a part of an embodiment of the present disclosure is a useful machine operation. Embodiments of the present disclosure also relate to devices or equipment for performing these operations. The device may be specially constructed for the desired purpose, or the device may be a general-purpose computer selectively activated or configured by a computer program stored in a computer. Specifically, various general-purpose machines may be used together with computer programs written according to the teachings of this article, or it may be more convenient to construct more specialized equipment to perform the desired operations.
[0159] The present disclosure may also be embodied as computer-readable code on a computer-readable medium. A computer-readable medium is any data storage device that can store data, which can then be read by a computer system. Examples of computer-readable media include hard drives, network attached storage devices (NAS), read-only memories, random access memories, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. The computer-readable medium may include a computer-readable tangible medium distributed on a network-coupled computer system so that the computer-readable code is stored and executed in a distributed manner.
[0160] Although the method operations are described in a particular order, it should be understood that other housekeeping operations may be performed between the operations, or the operations may be adjusted so that they occur at slightly different times, or the operations may be distributed in a system that allows processing operations to occur at various intervals associated with the processing as long as the processing of the superimposed operations is performed in the desired manner.
[0161] Although the foregoing disclosure has been described in some detail for the purpose of clarity of understanding, it will be apparent that some changes and modifications may be practiced within the scope of the appended claims. Therefore, the embodiments of the present invention should be considered illustrative rather than restrictive, and the embodiments of the present disclosure are not limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.
Claims
1. A method comprising: training an artificial intelligence (AI) model for a video game to play the first scenario of the video game using training state data and associated success criteria collected from a plurality of game play sessions of a plurality of players playing the first scenario of the video game, wherein the video game includes one or more scenarios and the one or more scenarios are key points in the game play sessions and / or involve completing intermediate tasks, and wherein the training state data includes metadata associated with the game play sessions; receiving input state data during the player's game play of the first instance; applying the input state data to the AI model to generate an output indicative of a predicted degree of success of the game play for the first scenario; performing analysis of said output based on predefined objectives including providing guidance; identifying a weakness of the player controlling the game play by determining that the input state data has a below average level of success; and Based on the analyzed output, an action is performed to achieve the predefined goal, the action comprising executing a training session corresponding to the identified weakness for delivery to the player's client device.
2. The method according to claim 1, further comprising: Applying the input state data to the AI model to generate next input state data; as well as including said next input state data in the recommendation, Wherein the next input state data is configured to play the game through the first scenario so as to advance the player's game play through the video game.
3. The method according to claim 1, further comprising: The associated success criterion is adjusted based on the training state data to obtain incrementally better results when playing the first scenario.
4. The method according to claim 1, in, The associated success criterion is used to distinguish similar training state data.
5. The method according to claim 1, wherein: Training the AI model includes: executing multiple instances of a video game supporting the multiple game play sessions; collecting training status data associated with the plurality of game play sessions; defining said associated success criteria; and The training state data and the associated success criteria are provided to a deep learning engine.
6. The method according to claim 1, further comprising: accessing a user profile of said player; as well as Recommendations are generated based on the user profile.
7. A non-transitory computer-readable medium storing a computer program for executing a method, the computer-readable medium comprising: Program instructions for training an artificial intelligence (AI) model for a video game to play a first scenario of the video game using training state data and associated success criteria collected from a plurality of game play sessions of a plurality of players playing the first scenario of the video game, wherein the video game includes one or more scenarios and the one or more scenarios are key points in the game play sessions and / or involve completing intermediate tasks, and wherein the training state data includes metadata associated with the game play sessions; program instructions for receiving input state data during the player's game play of said first context; program instructions for applying the input state data to the AI model to generate an output indicative of a predicted degree of success of the game play for the first scenario; program instructions for performing analysis of said output based on predefined objectives including providing guidance; Program instructions for identifying a weakness of the player controlling the game play by determining that the input state data has a below average level of success; and Program instructions for performing actions based on the analyzed output to achieve the predefined goal, the actions comprising executing a training session corresponding to the identified weakness for delivery to the player's client device.
8. The non-transitory computer-readable medium of claim 7, further comprising: program instructions for applying the input state data to the AI model to generate next input state data; and program instructions for including said next input state data in a recommendation, Wherein the next input state data is configured to play the game through the first scenario so as to advance the player's game play through the video game.
9. The non-transitory computer-readable medium of claim 7, further comprising: Program instructions for adjusting the associated success criterion based on the training state data to obtain incrementally better results when playing the first scenario.
10. The non-transitory computer readable medium according to claim 7, in, In the method, the associated success criterion is used to distinguish similar training state data.
11. The non-transitory computer readable medium of claim 7, wherein: The program instructions for training the AI model include: program instructions for executing a plurality of instances of a video game supporting the plurality of game play sessions; program instructions for collecting training status data associated with the plurality of game play sessions; program instructions for defining said associated success criteria; and Program instructions for providing the training state data and the associated success criteria to a deep learning engine.
12. The non-transitory computer-readable medium of claim 7, further comprising: program instructions for accessing a user profile of said player; and Program instructions for generating recommendations based on the user profile.
13. A computer system comprising: processor; A memory coupled to the processor and having instructions stored therein, which, if executed by the computer system, causes the computer system to perform a method for implementing a graphics pipeline, comprising: training an artificial intelligence (AI) model for a video game to play the first scenario of the video game using training state data and associated success criteria collected from a plurality of game play sessions of a plurality of players playing the first scenario of the video game, wherein the video game includes one or more scenarios and the one or more scenarios are key points in the game play sessions and / or involve completing intermediate tasks, and wherein the training state data includes metadata associated with the game play sessions; receiving input state data during the player's game play of the first instance; applying the input state data to the AI model to generate an output indicative of a predicted degree of success of the game play for the first scenario; performing analysis of said output based on predefined objectives including providing guidance; identifying a weakness of the player controlling the game play by determining that the input state data has a below average level of success; and Based on the analyzed output, an action is performed to achieve the predefined goal, the action comprising executing a training session corresponding to the identified weakness for delivery to the player's client device.
14. The computer system according to claim 13, wherein the method further comprises: Applying the input state data to the AI model to generate next input state data; as well as including said next input state data in the recommendation, Wherein the next input state data is configured to play the game through the first scenario so as to advance the player's game play through the video game.
15. The computer system according to claim 13, the method further comprising: The associated success criterion is adjusted based on the training state data to obtain incrementally better results when playing the first scenario.
16. The computer system according to claim 13, in, In the method, the associated success criterion is used to distinguish similar training state data.
17. The computer system of claim 13, wherein in the method, training the AI model comprises: executing a plurality of instances of a video game supporting the plurality of game play sessions; collecting training status data associated with the plurality of game play sessions; defining said associated success criteria; as well as The training state data and the associated success criteria are provided to a deep learning engine.
Citation Information
Patent Citations
Game machine
CN101254346A
Multiplayer video game matchmaking system and methods
CN107970612A
Realtime dynamic modification and optimization of gameplay parameters within a video game application
CN108499108A
Mission-based, game-implemented cyber training system and method
US20170304707A1
System and method for conducting a game including a computer-controlled player
US7892080B1
