Training artificial intelligence (AI) models using cloud gaming networks

Abstract

The network of the game cloud system's server console is used to collect training data related to remote players. [Solution] A method for processing an artificial intelligence (AI) model for a game application. The method includes training an AI model from multiple gameplays of a game application scenario using training state data collected from multiple gameplays of the scenario and the associated success criteria for each of the multiple gameplays. The method includes receiving first input state data during a first gameplay of the scenario. The method includes applying the first input state data to the AI ​​model to generate an output indicating the success level of the scenario for the first gameplay. The method includes performing an analysis of the output based on a predefined objective. The method performs an action to achieve the predefined objective based on the analyzed output.

Description

Technical Field

[0001] The present disclosure relates to the training of an artificial intelligence (AI) model, and more particularly, to collecting training data related to remote players using the network of a server console of a game cloud system, where the training data is used for training the AI model.

Background Art

[0002] Artificial intelligence algorithms are designed to learn from data. An AI model can be constructed based on the learned data and has sufficient flexibility to perform multiple functions according to the input given to the AI model.

[0003] However, providing data for training an AI model is complex. In a non-trivial example, an AI model configured to recognize objects is trained using an enormous number of object images. For example, an enormous number of object images are used for training the AI model. The generation and collection of these images are difficult and very time-consuming. Basically, an AI model is trained to recognize every existing type of object. Imagine trying to collect multiple images of that object taken from different viewpoints for each object. In this way, when a new image of an object is presented to the AI model, the AI model can extract various identification characteristics (such as contours, colors, features, sizes, etc.) and determine whether those characteristics match the characteristics of the learned objects. The number of objects is infinite, and the various views of those objects are also infinite. Therefore, the training of an AI model for object recognition can be an ongoing process.

[0004] Embodiments of the present disclosure have been made under such a background.

Summary of the Invention

[0005] Embodiments of this disclosure relate to a system and method for training an AI model related to the gameplay of a game application. The AI ​​model is built via a deep learning engine and is configured to provide various functionalities related to the game application and / or the gameplay of the game application (e.g., providing recommendations, discovering player weaknesses, discovering defects in the game application, training players, providing opponents to players, etc.). The AI ​​model can be trained using a vast network of servers (e.g., server consoles, game servers, etc.) configured to run instances of the game application that support multiple gameplays. For example, a server network may support remote user gameplay, with each remote user playing the game application via a corresponding client device. A standard process for supporting remote user gameplay might involve collecting training data from the server network and using it to train an AI model related to the game application. By collecting training data from an existing network of servers configured to support remote user gameplay, the time and effort required to collect training data are centralized and reduced. In other words, recording across the entire server network significantly reduces the time required to build the input set of training data. Partially, the AI ​​model learns the complex details of the game application and how to play it. In this way, given a gameplay situation of the game application (e.g., the game state of a particular gameplay) as input, the AI ​​model can analyze the situation and provide an appropriate response. The response may depend on a predefined objective. Because the AI ​​model understands how to play the game application with various objectives (e.g., exploratory, straightforward to the end, easiest play, most difficult play, etc.), it can determine how to direct the gameplay (e.g., determine what the next input sequence is needed to achieve the given objective). Specifically, AI models can be used to perform various functionalities related to game applications and / or gameplay, the functionalities of which depend on the corresponding predefined purpose. For example, an AI model can be used to train professional gamers to become the best among all gamers by having trainee gamers compete against ultimate opponents, or by guiding trainee gamers through various tasks to improve their weaknesses. Furthermore, AI models can be continuously improved by continuously collecting training data and comparing new training data with existing training data based on success criteria.

[0006] In one embodiment, a method for AI training and AI applications, more specifically, a method for processing an AI model for a game application, is disclosed. The method includes training an AI model from multiple gameplays of a scenario for a game application using training state data collected from multiple gameplays of a scenario and the associated success criteria for each of the multiple gameplays. The method includes receiving first input state data during a first gameplay of the scenario. The method includes applying the first input state data to the AI ​​model to generate an output indicating the success level of the scenario for the first gameplay. The method includes analyzing the output based on a predefined objective. The method includes taking action to achieve the predefined objective based on the analyzed output.

[0007] In another embodiment, a non-temporary computer-readable medium is disclosed for storing computer programs for AI training and applications. This computer-readable medium includes program instructions for processing an AI model for a game application. The computer-readable medium includes program instructions for training an AI model from multiple gameplays of a scenario for a game application, using training state data collected from multiple gameplays of a scenario and the associated success criteria for each of the multiple gameplays. The computer-readable medium includes program instructions for receiving first input state data during a first gameplay of a scenario. The computer-readable medium includes program instructions for applying the first input state data to the AI ​​model to generate an output indicating the success level of the scenario for the first gameplay. The computer-readable medium includes program instructions for performing an analysis of the output based on a predefined objective. The computer-readable medium includes program instructions for performing actions to achieve a predefined objective based on the analyzed output.

[0008] In yet another embodiment, a computer system is disclosed which includes a processor and a memory coupled to the processor and containing instructions that, when executed by the computer system, cause the computer system to execute a method for processing an AI model for a game application. The method executed by the computer system includes training an AI model from multiple gameplays of a scenario for a game application using training state data collected from multiple gameplays of the scenario and the associated success criteria for each of the multiple gameplays. The method includes receiving first input state data during a first gameplay of the scenario. The method includes applying the first input state data to the AI ​​model to generate an output indicating the success level of the scenario for the first gameplay. The method includes analyzing the output based on a predefined objective. The method includes taking action to achieve the predefined objective based on the analyzed output.

[0009] In another embodiment, a method for training an AI is disclosed. This method involves running multiple instances of a game application on multiple servers, which support multiple gameplays of the game application. This method involves collecting training state data on multiple servers related to corresponding gameplays of scenarios in the game application. This method involves defining success criteria for the training state data. This method involves training an AI model for the scenarios by providing the training state data and success criteria to a deep learning engine, the trained AI model providing multiple outputs for multiple inputs.

[0010] Other aspects of this disclosure will become apparent from the following embodiments for carrying out the invention, in conjunction with the accompanying drawings illustrating the principles of this disclosure as an example.

[0011] This disclosure can be best understood by referring to the following description in conjunction with the attached drawings. [Brief explanation of the drawing]

[0012] [Figure 1A] One embodiment of the present disclosure is a system for training an artificial intelligence (AI) model via a network of backend servers running instances of a game application, wherein the AI ​​model is used to provide various functionalities related to the game application. [Figure 1B] An exemplary neural network used to construct an AI model according to one embodiment of this disclosure is shown. [Figure 2A] A system diagram 200 for providing gaming via a cloud gaming network according to one embodiment of this disclosure is shown. [Figure 2B] One embodiment of the present disclosure describes a system that provides gaming to multiple players of a game application running via a cloud gaming network and collects training data from the players' gameplay for use in training an AI model, wherein the AI ​​model is configured to provide various functionalities related to the game application and / or the gameplay of the game application. [Figure 3A] This flowchart illustrates a method for training an AI model over a network of backend servers running instances of a game application, according to one embodiment of the present disclosure, wherein the AI ​​model is used to provide various functionalities related to the game application and / or gameplay of the game application. [Figure 3B-1] A data flow diagram illustrating a process for training an AI model over a network of backend servers running instances of a game application, according to one embodiment of the present disclosure, wherein the AI ​​model is used to provide various functionalities related to the game application and / or gameplay of the game application. [Figure 3B-2]A data flow diagram illustrating a process for training an AI model over a network of backend servers running instances of a game application, according to one embodiment of the present disclosure, wherein the AI ​​model is used to provide various functionalities related to the game application and / or gameplay of the game application. [Figure 3C-1] This is a screenshot of gameplay from a game application according to one embodiment of the present disclosure, illustrating a type of success criterion that may be used to train an AI model that understands how to play the game application. [Figure 3C-2] This diagram illustrates the collection of training state data from multiple gameplays according to one embodiment of the present disclosure, with multiple instances running on multiple servers supporting those gameplays. [Figure 4A] This flowchart illustrates a method for applying an AI model configured to understand how to play a game application, according to one embodiment of the present disclosure, wherein the AI ​​model is used to provide various functionalities related to the game application and / or gameplay of the game application. [Figure 4B] A block diagram of an AI processor engine configured to apply an AI model configured to understand how to play a game application, according to one embodiment of the present disclosure, wherein the AI ​​model is used to provide various functionalities related to the game application and / or gameplay of the game application. [Figure 5A] The present disclosure shows a data flow diagram illustrating a process for providing recommendations to a player playing a game application, wherein the recommendations are provided by an AI model trained over a network of backend servers running instances of the game application. [Figure 5B]This data flow diagram illustrates a process, according to one embodiment of the present disclosure, of balancing teams of players playing a game application using player profiles determined by an AI model trained over a network of backend servers running instances of the game application. [Figure 5C] The present disclosure shows a data flow diagram illustrating a process for training an AI model via autoplay directed by the AI ​​model, wherein the AI ​​model is trained via a network of backend servers running instances of a game application. [Figure 5D] This data flow diagram illustrates a process that automatically navigates within a game application using an autoplayer, which is directed by an AI model trained over a network of backend servers running instances of the game application, according to one embodiment of the present disclosure. [Figure 5E] One embodiment of the present disclosure shows a data flow diagram illustrating a process for providing an opponent to a player, the opponent being directed by an AI model trained over a network of backend servers running instances of a game application. [Figure 5F] A data flowchart illustrating a process that provides various services for identifying player weaknesses and training players to overcome those weaknesses, according to one embodiment of the present disclosure. [Figure 6] The following are exemplary device components that can be used to perform various embodiments of the present disclosure. [Modes for carrying out the invention]

[0013] The following detailed description includes many specific details for purposes of illustration, but those skilled in the art will recognize that many modifications and variations to the following details are within the scope of the present disclosure. Accordingly, the aspects of the present disclosure described below are presented without loss of generality with respect to the claims following this description and without imposing limitations on those claims.

[0014] Broadly speaking, various embodiments of the present disclosure describe systems and methods that implement deep learning (also referred to as machine learning) techniques to construct a game application and / or an AI model related to the gameplay of a game application. Specifically, the AI model is configured to provide various functionalities including, in relation to a game application and / or the gameplay of a game application, predicting and / or determining actions to be taken in response to a given situation (e.g., game state) of the game application. For example, the AI model can be used to train a professional gamer to be the best in the entire game. The AI model is trained and / or constructed using a network of servers that execute instances of a game application that support one or more gameplays (e.g., via a cloud gaming system). The AI model may be trained based on success criteria, such as by following one path by another similar path via a more successful AI model from the perspective of the success criteria. That is, the AI model learns to follow a more successful path. Further, the AI model can be continuously improved by continuously collecting training data, comparing new training data with existing training data, and selecting the best training data based on success criteria.

[0015] Having generally understood the various embodiments as described above, detailed examples of the embodiments will be described with reference to the various drawings.

[0016] FIG. 1A is a system 100A for training an artificial intelligence (AI) model 160 via a network of a backend server (e.g., a game cloud system) that executes an instance of a game application according to an embodiment of the present disclosure. The AI model is used to provide various functionalities related to the game application and / or the gameplay of the game application according to an embodiment of the present disclosure. Specifically, the trained AI model 160 can be implemented and / or supported by a backend server via a network 150 (e.g., the Internet), and this backend server provides artificial intelligence and / or deep learning, machine learning (e.g., via a deep, machine, learning engine 190) for building and applying the trained AI model 160 related to the game application and / or the gameplay of one or more players located at one or more locations around the world. The trained AI model 160 is trained to learn the complex details of the game application and / or the complex details of the gameplay of the game application (e.g., through the corresponding gameplay). In this way, when the gameplay situation of the game application (e.g., the game state of a specific gameplay) is given as input, the AI model can analyze the situation and provide an appropriate response to the situation. The response may depend on a predefined purpose (e.g., performing assistance, etc.). For example, the trained AI model 160 can determine how the game application should respond or how the player should respond during the corresponding gameplay.

[0017] As illustrated, multiple client devices 230 (e.g., devices 230a to 230n) are connected to a backend game cloud system (GCS) 201 via a network 150 (e.g., the internet) to support multiple gameplays of a particular game application. For example, multiple remote players are playing the game application through their respective client devices 230. A client device can be any type of computing device having at least memory and a processor module, which can connect to a backend server system (e.g., GCS 201) via the network 150. The client device is configured to interact with an instance of the corresponding game application running locally or remotely, via input commands used to drive the gameplay, in order to perform the gameplay of the corresponding player.

[0018] GCS201 includes multiple servers 205 (e.g., servers 205a to 205n) that run multiple instances (e.g., instances 110a to 110n) of the game application 110. For example, server 205a is configured to run instance 110a of the game application that supports corresponding gameplay for corresponding players via corresponding client devices. Each of the servers 205 may be configured to have at least memory and a processor module that can run the game application via corresponding instances of the game application that support corresponding gameplay. For example, each server 205 may be a server console, a game console, a computer, etc. Each server 205 is configured to stream data 105 (e.g., rendered images and / or frames of the corresponding gameplay) back to the corresponding client device via the network 150. In this way, computationally intensive game applications can continue to run on the backend server in response to controller input received and transmitted by the corresponding client device. Each server is capable of rendering images and / or frames, then encoding (e.g., compressing) them, and streaming them to the corresponding client device for display.

[0019] In one embodiment, GCS201 includes a distributed game engine system and / or architecture. Specifically, a distributed game engine that executes game logic is configured as an instance of the corresponding game application. Generally, the distributed game engine takes in each function of the game engine and distributes those functions to be executed by a number of processing entities. Individual functions can be further distributed across one or more processing entities. Processing entities can be configured in various ways, such as metal or physical hardware, and / or as virtual components or virtual machines, and / or as virtual containers. A container is different from a virtual machine because it virtualizes an instance of a game application that runs on a virtualized operating system. Processing entities may utilize and / or rely on servers and their underlying hardware on one or more servers (compute nodes) of GCS201, and the servers may be located on one or more racks. The distributed synchronization layer coordinates, assigns, and manages the execution of these functions for various processing entities. In this way, the execution of these functions is controlled by the distributed synchronization layer, enabling the generation of media for the game application (e.g., video frames, audio, etc.) in response to controller input from the player. The distributed synchronization layer enables the efficient execution of critical game engine components / functions across distributed processing entities (e.g., via load balancing) so that these functions are distributed and reconfigured for more efficient processing.

[0020] These various functions performed by a game engine include fundamental processor-based capabilities for running game applications and services associated with them. For example, processor-based capabilities include 2D or 3D rendering, physics, physics simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, ray tracing, shadowing, culling, transformation, and artificial intelligence. Furthermore, services for game applications include streaming, encoding, memory management, multithreading management, quality of service (QoS), bandwidth testing, social networking, social friend management, communication with friends on social networks, communication channels, texting, instant messaging, and chat support.

[0021] Furthermore, the distributed synchronization layer can be easily scaled (on individual distributed game engines) by adding more processing entities to handle increasing processing demands or processing complexity. In other words, processing capacity can be elastically increased or decreased based on demand. This distributed game engine system can also be scaled across many users, with each user being supported by a corresponding distributed game engine in a game application's multiplayer game session, for example. Therefore, this multiplayer game session is not constrained by hardware limitations that limit the maximum number of players in a session due to performance issues (e.g., latency). Instead, this scalable distributed game engine system can scale up or down the number of distributed game engines according to the number of players participating in a multiplayer game session, without any hardware constraints. As a result, thousands of players could potentially participate in a single multiplayer game session.

[0022] System 100A can build (e.g., train) an AI model 160 using a network of servers configured to run instances of a game application that support multiple gameplays. Specifically, training data 345 is collected from the running instances of the game application on multiple servers 205. More specifically, the collection for use as training data is done without the knowledge of any of the remote players associated with the various gameplays of the game application. The training data 345 includes controller inputs 236 used to direct corresponding gameplay, game states 133 of instances of the game application occurring during gameplay, response metrics, success criteria 135, success metrics, and additional analysis performed on the gameplay. In some cases, the training data 345 may be generated by instances of the game application (e.g., game state metadata). The training data 345 is provided as input to the deep learning and / or machine learning engine 190. The deep learning engine 190 includes a modeler 120 configured to build and / or train an AI model 160 using the training data 345, as will be further described below in relation to Figure 1B, and Figures 3A, 3B-1, and 3B-2.

[0023] The AI ​​model 160 may be implemented during subsequent gameplay of the game application (e.g., by a player, by an automated player, etc.) (e.g., after the AI ​​model has been trained). For example, the AI ​​model may be implemented and / or run on a backend server that supports gameplay, and gameplay may run on a local device (for the player) or on the backend server. For example, the trained AI model may be implemented by a deep learning engine 190 to provide various functionalities to the gameplay of the game application.

[0024] As illustrated, the analyzer 140 is configured to utilize an AI model 160 trained to provide various functionalities related to the gameplay of a game application. Specifically, an input data stream 405 is provided as input to a deep learning engine 190 configured to implement the trained AI model 160. The trained AI model 160 provides an output in response to the input, the output of which depends on the predefined functionality and / or predefined purpose of the trained AI model 160. For example, the trained AI model 160 can be used by the analyzer 140 to determine what actions need to be taken during gameplay by the player or by a corresponding execution instance of the game application. The analyzer 140 includes an action generator 170 configured to perform actions in response to input state data 405 and taking into account the predefined objectives of the trained AI model 160. In this way, using the AI ​​model 160, the analyzer can provide a variety of functionalities, including providing services to a player playing the game application (e.g., providing recommendations, discovering player weaknesses, training players, providing opponents for players, discovering defects in the game application, etc.).

[0025] More specifically, the analyzer 140 is configured to perform various functionalities in relation to the game application and / or the gameplay of the game application. The analyzer 140 is configured to analyze the output from a trained AI model 160 to a given input (e.g., controller input 236, game state data 133, success criteria 135) and provide a response (e.g., an action). For example, the analyzer 140 may perform the following actions: provide a profile of the player playing the game application; provide recommendations to the player during gameplay of the game application (these recommendations may be structured taking into account the user profile); take over gameplay; use the player profile to build a fair and balanced team (e.g., a team that competes against each other in the game application); autoplay the game application for purposes such as automatically training an AI model; provide bot opponents; explore the game application; determine the weaknesses of the corresponding player and perform services to help the player overcome those weaknesses.

[0026] Figure 1B shows an exemplary neural network 190 (e.g., a deep learning engine) used to build a trained AI model that partially learns the complex details of a game application and how the corresponding game application is played. In this way, given a gameplay situation of the game application (e.g., a game state for a particular gameplay) as input, the AI ​​model can analyze the situation and provide an appropriate response to it. For example, the AI ​​model may be used to provide various functionalities, including predicting and / or deciding what actions to take in response to a given situation in the game application (e.g., a game state) in relation to the game application and / or the gameplay of the game application.

[0027] More specifically, the AI ​​model is trained and / or built using a network of servers running instances of a game application that support one or more gameplays (e.g., via a cloud gaming system). This training is performed by a deep learning engine 190 according to one embodiment of the present disclosure. In one embodiment, the neural network 190 may be implemented within an AI processor or engine 210 of a backend server. Specifically, the modeler 120 of the deep learning engine 190 in system 100A in Figure 1A is configured to learn everything about the game application to be used in subsequent gameplays of the game application by any player (e.g., real or virtual).

[0028] Specifically, the deep learning or machine learning engine 190, which works in conjunction with the modeler 120, is configured to analyze training data 345 collected by multiple backend servers configured to run multiple instances of the game application. The deep learning engine 190 utilizes artificial intelligence, including deep learning algorithms, reinforcement learning, or other AI-based algorithms, to build a trained AI model related to the game application and / or the gameplay of the game application. In this way, it can efficiently collect a vast amount of training data to partially assist in defining the game application and / or the gameplay of the game application, thereby including the context that the game application may present (for example, various scenarios and parts within those scenarios as defined within the trained AI model). Furthermore, the AI ​​model can be used (for example, by the analyzer 140 of the AI ​​processor engine 210 during or after training) to provide various functionalities in any subsequent gameplay of the game application, in relation to the game application and / or the gameplay of the game application. Therefore, the deep learning engine 190 can learn everything about the game application and / or the gameplay of the game application so that it can use the trained AI model to provide the player with the best possible service for subsequent gameplay of the game application. For example, the trained AI model can be used to train a professional gamer to become the world's best gamer, or to provide various tutorial sessions (e.g., video instruction, gameplay challenges) designed to address a player's weaknesses. The deep learning engine 190 may be configured to continuously improve the trained AI model given any updated training data. This improvement is based on determining the set of training data that can be used for training, based on how those sets function within a game application based on corresponding success criteria.

[0029] More specifically, during the learning and / or modeling phase, the training data 345 is used by the deep learning engine 190 to predict and / or determine how successful a particular gameplay of a game application will be given a set of input data. The resulting AI model of the game application can be used to determine what actions to take for a given gameplay of the game application given a set of input data. That is, the trained AI model can be used (e.g., by the analyzer 140) to provide various functionalities related to the game application and / or the gameplay of the game application. For example, the input data could be game state data (including, for example, controller inputs), and the trained AI model 160 could be used to generate a response to the input data. The response may be provided with or without user request or knowledge.

[0030] For example, the trained AI model 160 may be used by the analyzer 140 to provide recommendations to players playing a game application. The AI ​​model 160 may also be used by the analyzer 140 to create a player user profile specific to the game application or specific to the scenario of the game application. The AI ​​model 160 may be used by the analyzer 140 to control bot opponents within a game application, for example, when training a player against the world's best opponent created through training data, or when training a player against a virtual opponent of themselves (for example, playing against a virtual version of themselves) so that the user can improve step by step through self-play (and continuously improve through that self-play). AI model 160 may be used by analyzer 140 to discover player weaknesses and provide tutorial sessions to address those weaknesses. AI model 160 may also be used by analyzer 140 to discover problems within the game application (e.g., code vulnerabilities that lead to glitches). Further functionality is supported, although not necessarily described.

[0031] The neural network 190 constitutes an example of an automated analysis tool that analyzes a dataset to determine the complex details of game application gameplay, including responses and / or actions that can be determined and / or performed during gameplay of the game application. Various types of neural networks 190 are possible. In one example, the neural network 190 supports deep learning, which can be implemented by the deep learning engine 190. Thus, deep neural networks, convolutional deep neural networks, and / or recurrent neural networks can be implemented using supervised or unsupervised training. In another example, the neural network 190 includes a deep learning network that supports reinforcement learning (e.g., by using success criteria, success metrics, etc.) or reward-based learning. For example, the neural network 190 is set up as a Markov decision process (MDP) that supports reinforcement learning algorithms.

[0032] Generally, a neural network 190 represents a network of interconnected nodes, such as an artificial neural network. Each node learns information from data. Knowledge can be exchanged between nodes through interconnections. Input to the neural network 190 activates a set of nodes. This set of nodes then activates other nodes, thereby propagating knowledge about the input. This activation process is repeated across other nodes until an output is provided.

[0033] As illustrated, the neural network 190 includes a hierarchy of nodes. At the lowest level is the input layer 191. The input layer 191 includes a set of input nodes. For example, each of these input nodes is mapped to an instance of gameplay in a game application, and the instance includes one or more features that define that instance (e.g., controller input, game state, result data, etc.). Intermediate predictions of the model are determined through a classifier that creates labels (e.g., output, feature, node, classification, etc.).

[0034] At the top level is the output layer 193. The output layer 193 contains a set of output nodes. The output nodes represent decisions related to one or more components of the trained AI model 160 (e.g., actions, predictions, predictions of gameplay success for a given set of input data). As described above, the output nodes can identify predicted or expected actions, or learned actions, for a given set of inputs, and the inputs can define various scenarios or parts of scenarios for a game application. These results can be compared to predetermined and true results, or learned actions and results, obtained from gameplay used to collect training data, in order to iteratively determine appropriate predicted or expected responses and / or actions for a given set of inputs, by refining and / or modifying the parameters used by the deep learning engine 190. In other words, the nodes in the neural network 190 learn the parameters of the trained AI model 160 that can be used to make such decisions when refining the parameters.

[0035] Specifically, between the input layer 191 and the output layer 193, there is a hidden layer 192. The hidden layer 192 contains "N" hidden layers, where "N" is an integer greater than or equal to 1. Next, each hidden layer also contains a set of hidden nodes. The input nodes are interconnected with the hidden nodes. Similarly, the input nodes are not directly interconnected with the output nodes, as the hidden nodes are interconnected with the output nodes. If there are multiple hidden layers, the input nodes are interconnected with the hidden nodes of the lowest hidden layer. Next, these hidden nodes are interconnected with the hidden nodes of the next hidden layer, and so on. The hidden nodes of the next highest hidden layer are interconnected with the output nodes. An interconnection connects two nodes. The interconnection has numerical weights that can be learned, allowing the neural network 190 to adapt to the input and become learnable.

[0036] Generally, the hidden layer 192 allows knowledge about the input node to be shared among all tasks corresponding to the output node. To achieve this, in one implementation example, a transformation f is applied to the input node via the hidden layer 192. In one example, the transformation f is nonlinear. For example, various nonlinear transformations f are available, including the rectification function f(x) = max(0,x).

[0037] The neural network 190 also uses a cost function c to find the optimal solution. The cost function measures the deviation between the prediction output by the neural network 190, defined as f(x) for a given input x, and the ground truth or target value y (e.g., the expected outcome). The optimal solution represents a situation where there are no solutions with a cost lower than the cost of the optimal solution. An example of a cost function is the mean squared error between the prediction and the ground truth, given that ground truth labels are available data. During the learning process, the neural network 190 may employ various optimization methods using the backpropagation algorithm to learn model parameters (e.g., the interconnection weights between nodes in the hidden layer 192) that minimize the cost function. An example of such an optimization method is stochastic gradient descent.

[0038] In one example, the training dataset for neural network 190 might come from the same data domain. For instance, neural network 190 is trained to learn predicted or expected responses and / or actions to perform for a given set of inputs or input data. In this example, the data domain includes gameplay data collected through multiple gameplay sessions by multiple users to define baseline input data. In another example, the training dataset might come from various data domains to include input data other than the baseline.

[0039] Therefore, the neural network 190 can partially predict or determine the predicted or expected responses and / or actions to be performed for a given set of inputs (the state of a game application, such as a game state). Based on these predictions, the neural network 190 can also define a trained AI model 160, which is used to provide decisions on those results and / or actions to be performed when a set of inputs is given (e.g., various functionalities relating to the game application and / or gameplay of the game application).

[0040] Figure 2A shows a system 200A according to one embodiment of the present disclosure that supports the collection of training data used to build and / or train an AI model relating to a game application and / or the gameplay of the game application. Specifically, system figure 200A enables access to and play of video games stored in a game cloud system (GCS) 201. Generally speaking, the game cloud system GCS 201 may be a cloud computing system that operates via a network 220 and supports multiple players playing a game application through their corresponding gameplay. Data related to their gameplay may be provided as training data used to build and / or train an AI model relating to the game application and / or the gameplay of the game application. Specifically, system 200A includes the GCS 201, one or more social media providers 240, and client devices 230, all connected via a network 150 (e.g., the Internet). One or more user devices may connect to the network 150 to access services provided by the GCS 201 and the social media providers 240.

[0041] In one embodiment, the game cloud system 201 includes a game server 205, a video recorder 271, a tag processor 273, an account manager 274 including a user profile manager, a game selection engine 275, a game session manager 285, user access logic 280, a network interface 290, and a social media manager 295. The GCS201 may further include several game storage systems, such as a game state store, a random seed store, a user save data store, and a snapshot store, which may generally be stored in a data store 260. Other game storage systems may include a game code store 261, a recorded game store 262, a tag data store 263, a video game data store 264, and a game network user store 265. In one embodiment, the GCS 201 is a system that can provide game applications, services, game-related digital content, and interoperability between systems, applications, users, and social networks. The GCS 201 may communicate with user devices 230 and social media providers 240 via a social media manager 295 through a network interface 290. The social media manager 295 may be configured to associate one or more friends. In one embodiment, each social media provider 240 includes at least one social graph 245 that shows the user's social network connections.

[0042] Player / user 5 can access services provided by GCS201 via game session manager 285. For example, account manager 274 enables authentication and access to GCS201 by player 5. Account manager 274 stores information about member users / players. For example, each member user's user profile may be managed by account manager 274. In this way, member information may be used by account manager 274 for authentication purposes. For example, account manager 274 may be used to update and manage user information related to member users. Furthermore, game titles owned by member users may be managed by account manager 274. In this way, video games stored in data store 264 are available to all member users who own those video games.

[0043] In one embodiment, a user, for example, Player 5, can access services provided by GCS 201 and Social Media Provider 240 via a client device 230 through a connection over Network 150. The client device 230 can include any type of device, wired or wireless, portable or non-portable, having a processor and memory. In one embodiment, the client device 230 may be a smartphone, tablet computer, or a hybrid providing touchscreen functionality in a portable form factor. One exemplary device could be a portable phone device running an operating system and providing access to a variety of applications (apps) that are acquired over Network 150 and can run on a local portable device (e.g., a smartphone, tablet, laptop, desktop, etc.).

[0044] The client device 230 includes a display 232 that functions as an interface for player 5 to send input commands 236 and display data and / or information 235 received from GCS201 and social media provider 240. The display 232 may be configured as a touchscreen or as a display provided by a device typically having the capability to render a display, such as a flat panel display, a cathode ray tube (CRT), or another device. Alternatively, the client device 230 may have its display 232 separately from the device, similar to a desktop or laptop computer.

[0045] In one embodiment, the client device 230 is configured to communicate with the GCS 201 so that the player 5 can play a video game. For example, the player 5 can select an available video game from the video game data store 264 (e.g., by game title) via the game selection engine 275. The selected video game is then activated, loaded, and executed by the game server 205 on the GCS 201. In one embodiment, gameplay is performed primarily on the GCS 201 so that the client device 230 receives a stream of game video frames 235 from the GCS 201, and user input commands 236 to drive gameplay are returned to the GCS 201. The video frames 235 received from the streaming gameplay are displayed on the display 232 of the client device 230.

[0046] In one embodiment, after player 5 selects an available game title to play, a game session for the selected game title may be started by player 5 via the game session manager 285. The game session manager 285 first accesses the game state store in the data store 140 to retrieve the saved game state of the last session played by player 5 (for the selected game), and if available, allows player 5 to resume gameplay from the previous gameplay stop point. Once a resume or start point is identified, the game session manager 285 may notify the game execution engine in the game processor 210 to execute the game code for the selected game title from the game code store 261. After the game session has started, the game session manager 285 may pass game video frames 235 (i.e., streaming video data) to a client device, for example, client device 230, via the network interface 290.

[0047] During gameplay, the game session manager 285 may communicate with the game processor 210, the recording engine 271, and the tag processor 273 to generate or save a recording (e.g., video) of the gameplay or gameplay session. In one embodiment, the video recording of gameplay may include tag content and other game-related metadata that are input or provided during gameplay. Tag content may also be saved via snapshots. The video recording of gameplay may be saved in the recorded game store 262 along with any game metrics corresponding to that gameplay. Any tag content may be saved in the tag data store 263.

[0048] During gameplay, the game session manager 285 may communicate with the game processor 204 to deliver and receive user input commands 236 used to influence the outcome of corresponding gameplay in the video game. Input commands 236 entered by player 5 may be sent from the client device 230 to the game session manager 285 of the GCS201. Input commands 236 (e.g., controller inputs) that include input commands used to drive gameplay may also include user interactive input, such as tag content (e.g., text, images, video recording clips, etc.). Optional user play metrics (e.g., the time the user played the game) may also be stored in the user store of the game network, similar to game input commands. Multiple features available to the user can be enabled using selection information related to the video game's gameplay.

[0049] Figure 2B shows a system 200B, according to one embodiment of the present disclosure, which provides the collection of training data used to build and / or train an AI model 160 relating to a game application and / or the gameplay of the game application, wherein an instance of the game application supporting gameplay and / or instantiation of the game application runs on a cloud gaming network. Furthermore, system 200A is configured to support the implementation of an AI model 160 trained to provide various functionalities relating to the game application and / or the gameplay of the game application (e.g., providing recommendations, training players, discovering player weaknesses, providing bot opponents, etc.).

[0050] As illustrated, System 200B provides game control to multiple players 215 (e.g., players 5L, 5M...5Z) playing a game application run over a cloud game network, according to one embodiment of the present disclosure. In some embodiments, the cloud game network may be a game cloud system 210 including multiple virtual machines (VMs) running on a host machine's hypervisor, one or more of which are configured to run a game processor module using hardware resources available on the host hypervisor. In other embodiments, GCS201 includes a distributed game engine system and / or architecture that runs game logic, configured as corresponding instances of a game application. Generally, a distributed game engine takes in each function of the game engine and distributes those functions to be executed by a number of processing entities across one or more servers of GCS201. Individual functions can be further distributed across one or more processing entities. Referring to the drawings here, similar reference numbers indicate identical or corresponding parts.

[0051] As illustrated, the game cloud system 210 includes a game server 205 that provides access to multiple interactive video games or game applications. The game server 205 can be any type of server computing device available in the cloud and may be configured as one or more virtual machines running on one or more hosts. For example, the game server 205 may manage virtual machines that support game processors that instantiate instances of a user's game application. Therefore, the multiple game processors of the game server 205, which are associated with multiple virtual machines, are configured to run multiple instances of the game application, which are associated with the gameplay of multiple users 215. In this way, the backend server support provides streaming of the gameplay media (e.g., video, audio, etc.) of the multiple game applications to the corresponding multiple users. As described later, the training data collected from the multiple game processors running instances of the game application is used to build and / or train AI models related to the game application and / or the gameplay of the game application.

[0052] Multiple players 215 access the game cloud system 210 via the network 150, and players (e.g., players 5L, 5M...5Z) access the network 150 via their corresponding client devices 230'. The client devices 230' may be configured similarly to the client device 230 in Figure 1A, or they may be configured as thin clients that provide an interface with a backend server that provides computing capabilities. Specifically, the client device 230' of a corresponding player 5L is configured to request access to a game application via the network 150, such as the internet, and to render an instance of the game application (e.g., a video game) that is run by the game server 205 and delivered to the display device associated with the corresponding player 5L. For example, player 5L can continue to interact with an instance of a game application running on the game processor of the game server 205 via client device 230'. More specifically, the instance of a game application is executed by a game title execution engine 211 (e.g., a game engine) that runs the game logic 177 corresponding to the game application. The game logic (e.g., executable code) 177 that implements the game application is stored and accessible via the previously described data store 260, or game code store 261, or video game store 264, etc., and is used to execute the game application. The game title processing engine 211 can support multiple game applications using multiple game logics 177, as shown in the figure.

[0053] As described above, the client device 230' can receive input from various types of input devices 11, such as a game controller, tablet computer, keyboard, gestures captured by a video camera, mouse, or touchpad. The client device 230' can be any type of computing device having at least memory and a processor module, and can connect to the game server 205 via the network 150. The client device 230' of a corresponding player is also configured to generate rendering images executed by a remotely operating game title execution engine 211 and to display these rendering images on a display including a head-mounted display (HMD) 102. For example, the client device 230' is configured to interact with an instance of a corresponding game application that is executed remotely to perform the corresponding player's gameplay, such as via input commands used to drive gameplay.

[0054] In another embodiment, the multiplayer processing engine 119 described above provides control over a multiplayer game session of a game application. Specifically, when the multiplayer processing engine 119 is managing the multiplayer game session, the multiplayer session controller 116 is configured to establish and maintain communication sessions with each user and / or player within the multiplayer session. In this way, players within the session can communicate with each other under the control of the multiplayer session controller 116.

[0055] Furthermore, the multiplayer processing engine 119 communicates with the multiplayer logic 118 to enable player-to-player interaction within each player's respective game environment. Specifically, the state sharing module 117 is configured to manage the state of each player in a multiplayer game session. For example, the state data may include game state data that defines the gameplay state (of the game application) of a corresponding player (e.g., player 5L) at a specific point, as described above. Furthermore, the state data may also include user / player save data that includes information to personalize the video game for the corresponding player, as described above. For example, since the state data includes information associated with the user's character, the video game is rendered with a character that may be unique to that user (e.g., shape, appearance, clothing, weapons, etc.). In this way, the multiplayer processing engine 119, using the state sharing data 117 and the multiplayer logic 118, can overlay / insert objects and characters into each of the game environments of users participating in a multiplayer game session. This allows users within a multiplayer game session to interact with each other through their respective game environments (for example, those displayed on their screens).

[0056] Furthermore, support for a backend server via an AI processor 210, which may be integrated within GCS201 or located remotely from GCS201, may provide the construction and / or training of an AI model 160 related to the game application and / or the gameplay of the game application, and may also provide the implementation and / or application of the AI ​​model 160. Specifically, the backend AI processor 210 includes the deep learning engine 190 described above, configured to partially learn and / or model responses and / or actions (e.g., controller inputs) that will be performed on any given set of inputs (e.g., those defining the gameplay situation of the game application, including game states) in order to construct (e.g., via a modeler 120) and apply (e.g., via an analyzer 140) the trained AI model in relation to the game application and / or subsequent gameplay of the game application. For example, the modeler 120 within the deep learning engine 190 may operate to set parameters defined within the deep learning engine 190 that define various nodes of the input layer 191, hidden layer 192, and output layer 193, for the purpose of applying the trained AI model 160 within the deep learning engine 190. The modeler 120 can set the parameters of the AI ​​model 160 based on one or more success criteria used during training, as described above. In this way, the AI ​​model 160 is trained to learn the complex details of the game application and / or the complex details of playing the game application so that the AI ​​model 160 can be used to provide various functionalities related to the game application and / or the gameplay of the game application (e.g., predicting and / or deciding what action to take, including controller inputs, in response to a given situation such as a game state). Thus, the analyzer 140 can receive inputs that define the gameplay situation and analyze the output from the AI ​​model 160, and optionally inputs (e.g., input state data), in order to provide an appropriate response to the situation. The response here may depend on a predefined purpose (e.g., providing support, providing guidance, etc.).

[0057] Flowchart 300A, along with a detailed description of the various modules of System 100A and System 200B, discloses a method for training an AI model over a network of backend servers running instances of a game application, according to one embodiment of the present disclosure, the AI ​​model being used to provide various functionalities related to the game application and / or gameplay of the game application. Flowchart 300A may be implemented within the aforementioned backend servers (for example, within a game cloud system 201 combined with a deep learning engine 190).

[0058] Specifically, in 302, the method involves running multiple instances of a game application on multiple servers, with these multiple instances supporting multiple gameplays of the game application. For example, the multiple servers may be running within the game cloud system described earlier (e.g., GCS201), the game application is run remotely from the corresponding player, and media (e.g., video frames, audio, etc.) is streamed over the network to the player's client device. In this case, multiple gameplays of the game application are controlled by multiple players via corresponding client devices, which are remote from the server. In other embodiments, the game application runs locally for the user / player, and metadata from the running game application is delivered to a backend server over the network for analytical purposes (e.g., for training an AI model) or to support multiplayer game sessions. In yet another embodiment, multiple gameplays may be controlled automatically (e.g., via AI) (e.g., for self-training an AI model).

[0059] In 304, the method includes collecting training state data on multiple servers related to the corresponding gameplay of a game application scenario. The training state data includes metadata associated with the gameplay, which may include controller inputs, game state, gameplay progress, scenario outcome (e.g., success or failure), and user profile information. In other words, the training state data includes any data that may be relevant to understanding the game application and / or the gameplay of the game application. Since instances of the game application run on backend servers, access to the training state data is immediately available without the active participation of the players involved in that gameplay. In other words, players may not even be aware that training state data is being collected.

[0060] Training state data may include game state data that defines the state and / or circumstances of a game application at a specific point (e.g., during gameplay). For example, game state data may include game characters, game objects, attributes of game objects, game attributes, the state of game objects, and graphic overlays. In this way, game state data makes it possible to generate the game environment that existed at a corresponding point in the video game. Game state data may also include the state of any devices used to render gameplay, such as the state of the CPU, GPU, memory, register values, program counter values, programmable DMA state, buffer data for DMA, audio chip state, and CD-ROM state. Game state data can also identify which parts of the executable code need to be loaded in order to run the video game from that point. It is not necessary to capture and save all game state data; only enough data is needed for the executable code to start the game at the point corresponding to the snapshot.

[0061] Furthermore, the training state data may include user save data that personalizes the game application for the corresponding user. This includes information associated with the user's character, so that the video game is rendered with a character that may be unique to that user (e.g., shape, appearance, clothing, weapons, etc.). In this way, user save data enables the generation of a character for the corresponding user's gameplay, and that character has a state corresponding to the video game point associated with the snapshot. For example, user save data may include the game difficulty selected by the corresponding player when playing the game application, the game level, character attributes, character position, number of remaining lives, total number of usable lives, armor, trophies, and time counter values. User save data may also include user profile data that identifies the corresponding player.

[0062] Furthermore, the training state data may include random seed data generated via AI. While random seed data may not be part of the original game code, it can be added to the overlay to make the game environment appear more realistic and / or more engaging for the user. In essence, random seed data provides additional features to the game environment present at corresponding points in the player's gameplay. For example, AI characters may be randomly generated and presented as an overlay. AI characters are placed in the game environment to enhance the user experience and may or may not influence gameplay. For instance, in a city scene, such AI characters might randomly walk down the streets. Other objects may also be generated and presented as overlays. For example, background clouds and birds flying in space could be generated and presented as overlays.

[0063] A game application may contain one or more scenarios. A scenario can be a key point in the game application (e.g., a point necessary to progress in the game application), such as fighting a boss at the end of a level, or jumping over or passing through an object or obstacle blocking the only path to a destination (e.g., climbing a mountain, crossing a lake or river with aggressive crocodiles). A scenario may also be less important, such as completing an intermediate task in the game application. In these cases, a scenario may involve completing a task to obtain a reward (e.g., money, a valuable sword). Data collection for the purpose of training an AI model may be limited to data related to gameplay in one or more scenarios in question. In this way, the trained AI model uses the scenario-related data to understand the complex details of playing the game application through that scenario, without being contaminated by data that may not be relevant to playing that scenario.

[0064] Scenarios can be predefined by game application developers, for example. A scenario might be designed with a high level of difficulty, such that many players are expected to be unable to progress through it. In other cases, scenarios may be discovered through the analysis of collected training state data. That is, a particular part of a game application may prove difficult for players to complete during its corresponding gameplay. In such cases, that part of the game application can be identified as a scenario, and training state data can then be collected during gameplay of that identified scenario.

[0065] In 306, the method includes defining success criteria for training state data. For example, success criteria may be used by a deep learning engine for the purpose of training an AI model. Specifically, when training an AI model, success criteria may be applied to define the relationships between nodes in a layer (for example, by narrowing down the weights that define the relationships between two nodes in different layers). For example, success criteria may be used to distinguish similar training state data and gain some insight into how to play or choose how to play a game application through a scenario. In a simplified example, a scenario might involve the completion of a task (e.g., defeating a boss, avoiding obstacles), with two sets of training data each describing or outlining the process of how to play through the scenario (e.g., controller inputs, strategy). A success criterion can be used to determine which set of training state data is more successful in completing the task. For the purpose of training AI models related to game application scenarios and / or gameplay of game application scenarios, more successful sets of training state data may be weighted more heavily than other less successful sets of training state data (e.g., when defining interrelationships between nodes in the AI ​​model).

[0066] In 308, the method includes training an AI model for a scenario by providing training state data and success criteria to a deep learning engine, the trained AI model providing multiple outputs for multiple inputs. Since the training state data is scenario-related, the AI ​​model is trained to learn everything about the scenario of a game application and / or the gameplay of the scenario based on one or more success criteria. In this way, given a set of inputs related to subsequent gameplay of the scenario (e.g., game state, controller inputs, etc.), the AI ​​model can provide outputs that may be beneficial to that gameplay. In one embodiment, the output may indicate the success rate of the gameplay. That is, given the current state of the gameplay (e.g., the game state), the AI ​​model can predict where the gameplay is headed and how successful it will be in progressing through the corresponding scenario. As with further analysis of the output (e.g., by analyzer 140), analysis of the set of inputs (e.g., a set of current and past inputs) can also provide a response to the set of inputs. Specifically, actions may be taken as an output and in response to the set of inputs. For example, if the set of inputs indicates (e.g., via the output of the AI ​​model) that the gameplay is leading in a direction that will result in failure in progressing through the scenario, the output from the AI ​​model may be used (e.g., by analyzer 140) to provide recommendations or advice on how to proceed with situations encountered during gameplay of the game application scenario. If gameplay is associated with a remote player, recommendations may be provided without any prompting from the remote player so that a set of inputs is automatically provided to a trained AI model during gameplay. These recommendations may help the player successfully progress through a scenario, efficiently complete a scenario, play a scenario to acquire the most assets, or complete a scenario task that gives the player the best chance of progressing to later stages or scenarios in the game application. In other embodiments, recommendations are provided at the request of the remote player, and therefore, in response to the request, a set of inputs is provided to the AI ​​model.

[0067] Figures 3B-1 and 3B-2 show data flow diagrams illustrating a process for training an AI model over a network of backend servers running instances of a game application, according to one embodiment of the present disclosure. As described above, the AI ​​model has a great deal of knowledge about the game application and / or the gameplay of the game application so that, given input (e.g., something related to the state of the game application, i.e., the game state), the AI ​​model can be used to provide various functionalities related to the game application and / or the gameplay of the game application (e.g., providing assistance). The process in Figure 3B-2 provides different functionalities performed by Modeler 120' compared to the functionalities performed by Modeler 120 in Figure 3B-1, as described below.

[0068] As shown in Figures 3B-1 and 3B-2, multiple gameplays 310 (e.g., 310a-310n) of a game application are shown. To support the gameplay, instances of the game application may be running, which, as described above, run on backend servers such as a game cloud system. A game application may include one or more interesting scenarios. As shown in the figures, a game application includes a first scenario (S-1), a second scenario (S-2), and a third scenario (S-3). For example, a scenario may be one that is rated as difficult by the developer or discovered through the gameplay of the game application. In other examples, a scenario may be an intriguing part of a game application that is popular with players, and therefore players will want to know all about that scenario.

[0069] In one embodiment, multiple gameplays 310 are operated by multiple players P-1 to Pn via their respective client devices. In another embodiment, multiple gameplays 310 can be automatically controlled, for example, for the purpose of self-training an AI model using multiple backend servers. As illustrated, a gameplay yields various gameplay data 320a to 320n. The gameplay data may include metadata, including game state data, as described above. For example, game state data describes the state of the game at a particular point and may include controller input data. Furthermore, the gameplay data 320a to 320n may include records of gameplays 310a to 310n for the purpose of extracting metadata and / or training state data.

[0070] As shown in Figures 3B-1 and 3B-2, the capture engine 340 captures gameplay data 320a to 320n, as well as other data that may be provided, such as success criteria 330. As described above, success criteria may be used to distinguish similar training state data for the purpose of training the AI ​​model. Specifically, success criteria are used to train the AI ​​model so as to define the interrelationships between nodes in the layers within the AI ​​model, and the nodes may define features related to the game application and / or gameplay of the game application (e.g., controller input, game state, etc.). For example, success criteria may be used to determine which set of training state data is more successful, such as defining the weights of one or more paths (between nodes of one or more layers) through the AI ​​model. In this way, the AI ​​model can be used to give insights (e.g., strategies) about how to play or how to choose how to play a game application through a scenario. The ingested data is provided to the deep learning engine 190 as training state data 345.

[0071] As shown in Figures 3B-1 and 3B-2, the deep learning engine 190 includes a modeler 120 configured to train and / or build an AI model using training state data 345 based on one or more success criteria. The modeler 120 can implement artificial intelligence through various neural networks (e.g., convolutional, recurrent, etc.). The modeler 120 can implement AI by various algorithms, including, as examples for illustrative purposes, deep learning, reinforcement learning, supervised learning, unsupervised learning, reward-based learning (e.g., using success criteria, success metrics, etc.), incremental learning, etc. The deep learning engine 190 was previously introduced in Figure 1B. Specifically, the modeler 120 identifies a set of feature-dependent rules that make predictions given a set of inputs (e.g., features that can define the context or situation (game state) of a game application) when building an AI model. Predictions may include how successful a given set of inputs would be when playing a scenario. In this way, the AI ​​model can be used to determine what action to take when given a set of inputs.

[0072] As shown in Figures 3B-1 and 3B-2, the modeler 120 of the deep learning engine 190 includes a feature recognition engine 350 configured to identify multiple features of the training state data. For each gameplay of a corresponding scenario, the training state data contains features. For example, at a particular point in gameplay, instances of the training state data may be collected, and a training instance contains one or more features (e.g., a set of features for the training instance), and features may include variables, parameters, controller inputs, game state metadata, etc.

[0073] Therefore, the feature recognition engine 350 is configured to analyze training state data for the purpose of identifying and / or extracting features from the data. The feature recognition engine 350 may also be configured to learn features. In each training cycle via the deep learning engine 190, a training instance (e.g., a set of features) is provided as input, and the training instance may be associated with a specific point in the gameplay of a scenario. In this way, the deep learning engine 190 is configured to learn incrementally about the game application, the scenarios of the game application, and / or the gameplay of the scenarios of the game application.

[0074] As shown in Figure 3B-1, the modeler 120 is configured to learn rules that define relationships between features and outputs (e.g., predictions, actions, etc.), where features may be defined within one or more nodes located at one or more hierarchical levels of the AI ​​model 160 being trained. The modeler 120 constructs the AI ​​model 160 by linking features between layers such that a given set of data inputs leads to a specific output of the AI ​​model. Thus, the modeler 120 may be configured to generate features and / or nodes of the AI ​​model 160 as defined by rules that link features in various layers. For example, a rule can link one or more features or nodes (e.g., using relational parameters including weights) through the AI ​​model between inputs and outputs. In other words, one or more linked features make up a rule. The AI ​​model 160 contains a set of rules corresponding to outputs that are trained and each labeled or classified. A more detailed description of the functionality of the modeler 120 is provided in Figure 3B-2 below.

[0075] Specifically, as shown in Figure 3B-2, in the modeler 120, identified features identified and / or extracted from the input data by the feature identification engine 130 may be delivered to a classifier 360 configured to learn rules that define the relationship between features and outputs (e.g., predictions, actions, etc.). Features may be defined within one or more nodes located at one or more hierarchical levels of the AI ​​model under training. Each feature may be associated with one or more features of other layers, and one or more relational parameters define the interconnections between the first feature and other features of other layers of the AI ​​model (e.g., a second feature, a third feature, etc.).

[0076] For example, as shown in Figure 3B-2, the classifier 360 is configured to determine which label or output a set of features (a set that makes up a rule) belongs to. In other words, a rule links a given set of features, which may be defined as nodes in an AI model (i.e., nodes that describe training instances or points of gameplay in a game application scenario), to a specific output that is labeled by the classifier 360. For example, a rule can link one or more features or nodes (links or interrelationships between features defined via one or more relational parameters) between an input and an output via an AI model. The classifier may be configured to generate features and / or nodes in the AI ​​model 160, which are used to define the rule as described above. The output may be associated with a label generated, assigned, and / or determined by the classifier 360.

[0077] More specifically, the learned rules may be learned paths and / or learned patterns (e.g., through nodes or features of the AI ​​model leading to output nodes) for a given set of inputs and / or input data related to gameplay in a game application scenario. For example, one or more linked features and / or nodes make up the rules. The trained AI model is a set of rules and labels (i.e., outputs). In a supervised learning environment, the output is predetermined for a given set of features, and the deep learning engine learns rules that associate the set of features with the output (e.g., through labels). In an unsupervised learning environment, the given set of features is not automatically associated with an output, and rules may be learned by looking for similarities associated with other similar sets of features or by searching for clusters of data points. Depending on the success criteria defined for training, clusters may be preferred over other clusters. In either case, existing rules may be fitted to the input set of features, or new rules may be generated for the input set of features (likely similar to or an evolution of one or more existing rules). The output obtained according to the learned rules of the AI ​​model can predict how successful a corresponding set of input features might be when used to play a game application scenario. Furthermore, the output from the AI ​​model (and optionally, the set of input features) may be used (e.g., via an analyzer) to determine a set of actions to take at a particular point in the gameplay of the scenario (determined by the game application situation or game state indicated by the set of input data). For example, an action could include a set of controller inputs that would be suggested as recommendations, or control commands for a game application to respond to the inputs. One or more rules may be generated for a given set of features, or a set of similar features. Depending on the corresponding success or reward criteria, one rule may be preferred for a given set of features, or a set of similar features. For a given set of features, the most successful output may be selected for that set of features. In other words, for a given set of features (e.g., inputs), the most successful rule (and output) is selected and used within the trained AI model 160, which forms the basis of incremental learning.

[0078] In a simple example, success criteria can be defined by the points earned, with the path that generates the most points (e.g., associated labels) being considered more successful. In another example, success criteria may relate to the level of the player character's or opponent's life bar, as further described in Figure 3C-1. Other success criteria may be defined based on the player. For example, by identifying an expert by one or more success criteria, that expert's corresponding gameplay can be targeted and used as training state data, generally to learn the best action to take for a given state or situation of gameplay in a game application scenario. In this way, success criteria can be defined to determine a player's skill level. This may include the speed of the player's response time, the accuracy of the player's aiming at one or more targets (e.g., generally, skilled players have fast triggers and move quickly, decisively, and accurately from one target to another), and the speed of the cycle between controller inputs.

[0079] As shown in Figures 3B-1 and 3B-2, the modeler 120 constructs and / or outputs a trained AI model 160 that links learned paths and / or learned patterns (e.g., combined labels of the AI ​​model) to a given set of inputs and / or input data related to gameplay in a game application scenario. The AI ​​model 160 can then be used to provide one or more functionalities related to the game application and / or gameplay of the game application. In other words, given a set of inputs that can represent the subsequent gameplay situation by the player, the output obtained as a result of the trained AI model 160 can be used (for example, via an analyzer) to predict and / or determine the best set of actions to take at that particular point in the gameplay of the scenario defined by the corresponding set of input data. For example, after the AI ​​model 160 has been trained, a player may be playing a scenario in a game application. The player may also be experiencing difficulties progressing through the scenario, and this may be reflected in the AI ​​model's output. New and subsequent input state data (e.g., game state) may relate to any data related to that particular point in the player's gameplay (where difficulties are experienced). The input state data for that scenario may be received and provided to the AI ​​model via the deep learning engine 190, and the AI ​​model may output a prediction of how successful the gameplay will become when it plays the scenario given a given situation in the game application. The output from the AI ​​model can be analyzed and used to perform various functionalities related to the game application and / or gameplay of the game application. For example, the output may be analyzed to determine the best set of actions to take at a particular point in the gameplay of a scenario. Actions can be performed based on the output. For example, the trained AI model 160 can provide the player with recommendations for how to advance their gameplay.

[0080] Based on the output of the AI ​​model 160 for a given set of input data, other functionalities may also be determined and generated, manifesting themselves in the form of outputs or actions. In other words, input state data is received during the initial gameplay of the scenario and provided as input to the deep learning engine. The deep learning engine applies the AI ​​model to the input state data. Actions are performed based on an analysis of the AI ​​model's output, and these actions are performed in response to the input state data. Figures 4A to 4B illustrate the application of the AI ​​model to a given gameplay of the game application. Figures 5A to 5F provide various explanatory diagrams of the various actions or responses that can be performed depending on a predefined objective. Specifically, the AI ​​model implemented via the deep learning engine 190 matches a given input state data against one or more rules defined within the trained AI model (each rule provides associated or interconnected nodes and / or features). Each rule is associated with an output. Success criteria can be applied to generate rules. Furthermore, the analyzer 140 receives the output and performs additional analysis to determine appropriate actions in relation to the corresponding input data. For example, if a rule satisfies the success criteria with respect to a given set of input state data, a corresponding action can be identified and / or performed.

[0081] Figure 3C-1 is a screenshot 300C of gameplay from a game application according to one embodiment of the present disclosure, illustrating a type of success criterion that may be used to train an AI model to understand how to play the game application. Specifically, player P-1 controls the gameplay. As shown in the illustration, screenshot 300C shows a front view of a battle between Kratos 391 and enemy 396. In the "God of War" game application, Kratos is a Spartan warrior from Greek mythology tasked with killing Ares, the god of war. In gameplay, the player can control Kratos 391. Training state data 320a from gameplay related to screenshot 300C is supplied to the deep and / or machine learning engine 190 for the purpose of training the AI ​​model described above. Furthermore, the deep and / or machine learning engine 190 is supplied with success criteria 330, which are used by the deep learning engine 190 to train the AI ​​model. For example, success criteria can be used to distinguish paths through the AI ​​model (paths that traverse the nodes of the AI ​​model). In embodiments, one or more success criteria may be defined for use in training the AI ​​model. For example, success criteria may include gaining the most points, gaining the most assets or the strongest or most important assets.

[0082] In one example, success criterion 330 may be defined by whether the enemy's life bar 397 gets low during gameplay, with the life bar representing the corresponding character's health. Life bar 392 represents Kratos's health during gameplay. As illustrated, the enemy's life bar 397 is extremely low, which may indicate that the gameplay is successful or highly successful. In another example, the success criterion may be more refined and defined by how quickly the enemy's life bar 397 is depleted. If the life bar 397 is depleted very quickly, this indicates that the gameplay is being controlled by a highly skilled player. For example, a highly skilled player understands how to fight enemy 397, knows the sequence of controller inputs used to defeat enemy 396, and possesses the skill and / or ability to execute those sequences quickly and accurately (e.g., without deviation). When training an AI model, using a success criterion that emphasizes how quickly the life bar is depleted can help distinguish between successful and unsuccessful inputs and identify the sequence of controller inputs that should be used during specific situations or game states of gameplay that would be successful if executed by another player.

[0083] Figure 3C-2 illustrates the collection of training state data from multiple gameplays according to one embodiment of the present disclosure, with multiple instances running on multiple servers supporting the gameplays. It shows multiple gameplays 310 (e.g., 310a to 310n) of a game application. The multiple gameplays 310 are operated by multiple players P-1 to Pn via their respective client devices. As shown, the gameplays yield various gameplay data 320a to 320n, from which data extracted can be used as training state data 345. Gameplay data may include metadata, including game state data, as described above. Each gameplay represents playing scenario 1 (S-1) of the game application, and each gameplay is unique and yields unique results. That is, each gameplay can be associated with measuring the success of the progression of scenario S-1. This success may be partially measured using a success criterion 330, which, as described above, can be used by the modeler 120 of the deep and / or machine learning engine 190 to train the AI ​​model 160. For example, the success criterion may be used to distinguish similar training state data for the purpose of training the AI ​​model.

[0084] Flowchart 400A, along with a detailed description of the various modules of System 100A and System 200B, discloses a method for applying an AI model that knows everything about a game application and / or the gameplay of the game application (e.g., understands how to play the game application) based on one or more success criteria, according to one embodiment of the present disclosure, and this AI model is used to provide various functionalities related to the game application and / or the gameplay of the game application. Flowchart 400A may be implemented within the above backend server (e.g., within a game cloud system 201 combined with the deep learning engine 190).

[0085] Specifically, in 410, the method includes training an AI model from multiple gameplays of a game application scenario. For example, the AI ​​model is trained to play the game application scenario using training state data collected from multiple gameplays of the scenario, along with the relevant success criteria for each gameplay. In other words, the AI ​​model is trained to learn the complex details of the game application, and / or the complex details of playing the game application, or the scenario of the game application.

[0086] In one embodiment, the AI ​​model is trained using training state data collected across multiple gameplays of a game application, which are operated by multiple players via multiple client devices. For example, multiple servers may be running multiple instances of the game application, and these instances may support multiple gameplays. The training state data is collected by multiple servers and is associated with the corresponding gameplay of a scenario. For example, the training state data may include metadata associated with the gameplay, as described above, such as controller inputs, game states defining the state and / or circumstances of the game application at specific points during gameplay, gameplay progress, scenario outcomes (e.g., success or failure), and user profile information. In short, the training state data includes any data that may be relevant to understanding the game application and / or the gameplay of the game application.

[0087] Furthermore, one or more success criteria are defined for the training state data. For example, success criteria are used to train an AI model that understands how to play a game application. For example, success criteria may be used to define favorable links between nodes of the AI ​​model, or to define favorable paths through the nodes of the AI ​​model for a given set of input data. The success criteria and training state data are provided to a deep learning engine to train the AI ​​model.

[0088] In 420, the method includes receiving first input state data during a first gameplay of a scenario. In one embodiment, the first gameplay takes place after the AI ​​model has been trained; that is, the first gameplay is not used for training the AI ​​model. In another embodiment, the first gameplay may occur during training so that the first gameplay is used for training the AI ​​model (e.g., in a self-training mode). In yet another embodiment, the first gameplay cannot initially be used for training the AI ​​model and is primarily used to determine the appropriate action to take given the first input state data. The AI ​​model can then be improved by further training using the first gameplay (e.g., the first input state data).

[0089] In 430, the method includes applying first input state data to an AI model to generate an output. In one embodiment, the output may indicate the success rate of a scenario for a first gameplay. That is, the output can predict how successful the first gameplay will be in the progression of the scenario.

[0090] In 440, the method includes performing an analysis of the output based on a predefined objective. Furthermore, it is also possible to analyze a set of inputs (e.g., a set of current and past inputs). Depending on the predefined objective, the analysis can generate actions to be taken at specific points in the corresponding gameplay of a scenario (determined by the state of the game application or game state indicated by the set of input data). For example, if the predefined objective is to provide support, the analysis may generate recommendations or advice on how to proceed with situations encountered during gameplay of a game application scenario. If the predefined objective is to provide instruction, the analysis may identify the player's weaknesses and provide the player with a tutorial session to address those weaknesses. Other predefined objectives that are supported include providing game support, providing gameplay equivalence, automated training of AI models, detection of defects in game applications, and automated testing of game applications.

[0091] Therefore, in 450, the method includes taking actions to achieve a predefined objective based on the analyzed output. Figures 5A to 5F provide various explanatory diagrams and examples of various actions or responses that may be taken depending on the predefined objective.

[0092] Figure 4B is a block diagram of the AI ​​processor engine 210 of Figure 2B, configured to apply an AI model configured to understand how to play a game application, according to one embodiment of the present disclosure, wherein the AI ​​model is used to provide various functionalities related to the game application and / or gameplay of the game application. As shown, the AI ​​processor engine 210 includes a deep learning engine or machine learning engine 190 configured to train and / or apply an AI model 160, the AI ​​model 160 configured to provide an output for a given input. The AI ​​processor engine 210 also includes an analyzer for determining a response to an input using the output from the AI ​​model 160.

[0093] Specifically, input state data 405 is provided as input to a deep learning engine 190 configured to implement the AI ​​model 160. The input state data is associated with the gameplay of a game application and may include, as described above, controller inputs, game state data, user data, success criteria, etc. In some embodiments, the input state data 405 may be provided after the AI ​​model 160 has been trained. In other embodiments, the input state data 405 may be provided during the training of the AI ​​model 160 (e.g., self-training of the AI ​​model). As described above, the AI ​​model 160 generates an output 173 for a given input state data 405. For example, it can output how likely the gameplay is to succeed as the game application progresses through a scenario. The output can be analyzed by analyzer 140 to provide various functionalities related to the gameplay of the game application. As described above, analyzer 140 determines appropriate actions or responses according to predefined objectives (e.g., providing guidance or support). In other words, the AI ​​model 160 can be used by analyzer 140 to determine what is necessary to successfully guide the gameplay during a scenario. Therefore, analyzer 140 can determine and execute actions according to predefined objectives for given input state data. In this way, the analyzer can utilize AI model 160 to provide various functionalities related to the game application and / or the gameplay of the game application.

[0094] As illustrated, as part of the analysis, analyzer 140 may perform one or more functions. For example, the player profiler 144a of analyzer 140 is configured to profile a player playing a game application (e.g., to determine the player's skill level). The weakness identifyer 141a of analyzer 140 is configured to identify weaknesses in the player's game skills. The input control sequence parser 147a of analyzer 140 is configured to determine a sequence of controller inputs used by the player to control gameplay. The sequence map and / or route tracker 148a of the analyzer 140 are configured to track the progress of gameplay and to track progress within the game environment. For example, the tracker 148a may be configured to track the routes taken in the game environment, or to construct a map of the game environment, or to construct a node map of gameplay showing progress within the game application. Further modules are supported that provide analysis of the output of the AI ​​model for a given input state data 405.

[0095] The analyzer 140 is further configured to determine and execute an action by the action generator 170 in response to an analysis of the output determined in response to a given input state data 405. The action is determined and executed according to a predefined purpose. For example, the action may provide a service to a player playing a game application (e.g., providing a profile of the player playing the game application, providing recommendations to the player during gameplay of the game application (the recommendations may be structured taking into account the user profile), finding the player's weaknesses, providing services to address those weaknesses, training the player, providing the player with a bot opponent, taking over the player's gameplay, etc.). Actions may also provide game developers or other entities with services such as discovering defects in game applications, building equal teams using player profiles that compete against each other in game applications such as multiplayer online games, automatically learning AI models through autoplay, and exploring the game environment of game applications to discover defects. Figures 5A to 5F provide various explanatory diagrams of various actions or responses that may be performed depending on a predefined purpose.

[0096] Specifically, Figure 5A shows a data flow diagram illustrating a process for providing recommendations to a player playing a game application according to one embodiment of the present disclosure, wherein the recommendations are provided by an AI model 160 trained over a network of backend servers running instances of the game application. The diagram shows gameplay 501x ​​of the game application. Gameplay may be controlled by a player Px via a corresponding client device, and as described above, instances of the game application are running on backend servers of a streaming game service (e.g., a cloud gaming system). In other embodiments, the game application may run locally on the client device, and metadata is delivered to the backend servers for AI model support. Gameplay 501x ​​occurs during scenario S-1.

[0097] Input state data 505x from gameplay 501x ​​is provided to analyzer 140 configured to analyze the output of the trained AI model 160, which is implemented via deep learning engine 190 (in the application phase, not the training phase). The input state data 505x is received after the AI ​​model 160 has been trained and therefore may not be part of the training state data used to train the AI ​​model described above. The AI ​​model 160 is configured to provide an output 173, and analyzer 140 is configured to perform or provide an action to be taken based on an analysis of the output 173.

[0098] Specifically, the action generator 170 of the analyzer 140 includes a recommendation engine 145a configured to provide recommendations to the player during gameplay of a game application. For example, a predefined objective may be to provide the player Px with game support performed via the analyzer 140. Recommendations may be provided in response to the gameplay situation (e.g., game state), such as when the player is having difficulty successfully navigating a particular scenario of the game application. In one embodiment, the analyzer 140 can determine that the player is struggling by determining that a first input state data, or a feature within the first input state data, does not meet the corresponding success criteria. Success criteria may provide an indicator of how successful a gameplay will be, and in particular how successful a particular part of the gameplay will be. For example, success criteria may indicate how many points were accumulated during the scenario. If analyzer 140 determines from the output of AI model 160 that the first input state data does not meet the success criteria, it may provide recommendation 515 as an action to the player Px's client device.

[0099] In one embodiment, the recommendation may take into account the user / player profile 510x of player Px (e.g., the player's skill level), and the player profiler 144a may analyze output data and / or input state data from a trained AI model 160 to determine the user profile 510x for a scenario (e.g., how the player reacts to the game application during the scenario, skill level, and other player characteristic metrics). In other words, the recommendations reflect the player Px's skills. In other embodiments, the user profile 510x may be determined not only through past gameplay in the target game application but also through past gameplay in other game applications. For example, if the user profile 510x indicates that the player is an expert at the game, the recommendations 515 for player Px may provide small hints without giving too much detail. On the other hand, if the user profile 510x indicates that player Px is a beginner at the game, the recommendations 515 may provide detailed instructions for the player to follow.

[0100] In one embodiment, the recommendation 515 may include a query asking whether the player Px wants to use the autoplay function. If yes, the task autoplay engine 145b is configured to take over the player Px's gameplay. In this way, the task autoplay engine 145b automatically (and successfully) executes a scenario or a task within a scenario in the game application.

[0101] Figure 5B shows a data flow diagram illustrating the process of balancing a team of players playing a game application using player profiles determined by an AI model trained over a network of backend servers running instances of the game application, according to one embodiment of the present disclosure. Multiple gameplays 501 (e.g., 501a-501n) of the game application are shown. Gameplay can be controlled by multiple players P-1 to Pn via their corresponding client devices. As described above, instances of the game application run on a backend server of a streaming game service (e.g., a cloud gaming system). In other embodiments, the game application may run locally on the client device, and metadata is delivered to the backend server for AI model support. Gameplay 501a to 501n may occur, for example, through one or more scenarios S-1, S-2, and S-3.

[0102] Input state data 505a to 505n from gameplay 501a to 501n are provided to an analyzer 140 configured to analyze the output of a trained AI model 160, which is implemented via a deep learning engine 190 (in the application phase, not the training phase). In one embodiment, multiple input state data 505a to 505n may be received during a second set of gameplays of a game application scenario, which may be performed after the AI ​​model has been trained. The second set of multiple gameplay sequences is operated by multiple players, and the multiple input state data includes multiple player characteristic metrics for each player. The multiple input state data is applied to an AI model to generate multiple outputs representing multiple success levels for the second set of multiple gameplay sequences in the scenario. Furthermore, the multiple outputs are analyzed to determine multiple player profiles based on the multiple player characteristic metrics. The AI ​​model 160 is configured to provide the outputs, and the analyzer 140 is configured to perform or provide actions to be taken based on its analysis of the outputs. Specifically, the analyzer 140 is configured to build a balanced team of players from the multiple players based on their corresponding player profiles in order to achieve a predefined objective.

[0103] During the analysis, the analyzer 140 may perform user profiling using a player profiler 144a, etc. Specifically, the output from the trained AI model 160 is provided to the player profiler 144a, which is configured to provide profiles of players P-1 to Pn playing the game application. The player profiles 510a to 510n may be limited to gameplay in specific scenarios of the game application (for example, the profile determines or predicts how the corresponding player will react to the game application during the scenario, the corresponding player's skill level, etc.). For example, a player profile may be defined based on multiple player characteristic metrics, and each player profile includes one or more corresponding player characteristic metrics. Input state data can be received during gameplay of a game application by a player. The input state data includes multiple player characteristic metrics for multiple players. The input state data is also applied to the AI ​​model 160 as described above to determine the output. The player profiler 144a analyzes the output 173 and / or the input state data 505a~505n to determine the corresponding player profile for the corresponding player. For example, the corresponding player profile is based on corresponding player characteristic metrics provided as input to a trained AI model. For example, player characteristic metrics may include player accuracy, the rate at which the corresponding player generates input control data sequences, the corresponding player's reaction time to events in a scenario, the corresponding player's consistency, or the corresponding player's transition time between a first target and a second target. In other embodiments, the corresponding player profile may be determined not only from the target game application but also from past gameplay of other game applications, as described above.

[0104] Furthermore, the action generator 170 of the analyzer 140 includes a team balancer module 144b configured to build a fair and balanced team (for example, a team competing against each other in a game application) using player profiles, according to a predefined objective. For example, the predefined objective may be to provide equivalence to gameplay performed via the analyzer 140 (for example, in a massively multiplayer online game application (MMO)). For example, based on player profiles 510a-510n, the team balancer 144b is configured to create fairly balanced teams (e.g., teams competing against each other in a game application). In this way, the gameplay of the game application between the two teams becomes fair, engaging, and valuable for the players on each team. For example, balanced teams help to avoid one-sided games. Consider further dividing the player profiles into two groups (G1 and G2) based on skill levels in one or more categories. It is desirable that the players in each group, and / or the skills of the players, be evenly distributed across all teams (e.g., Team-1 to Team-n). In combat-focused game applications, each team will include one or more positions, such as leader, sniper, and trained (e.g., close combat) assassin. One explanatory diagram groups all skill positions and distributes them evenly across teams. In this diagram, leaders are excluded, and for simplicity, all leaders are selected from a single group G-2 so that all teams have equally competent leaders. All other skilled positions are assumed to be valued equally. In other words, each team should consist of several skilled positions, with equal numbers of players from each group G1 and G2. For example, each team includes one player from Group G1 and one player from Group G2 (excluding the leader). As shown in the diagram, Team-1 and Team-2 each include one player from Group G1 (a sniper) and one player from G2 (e.g., a killer). Similarly, Team-3 includes one player from Group G1 (a killer) and one player from G2 (e.g., a sniper). Therefore, each team is balanced according to user profiles (e.g., skill levels).

[0105] Figure 5C shows a data flowchart illustrating the process for training an AI model via autoplay directed by the AI ​​model 160, according to one embodiment of the present disclosure. As described above, the AI ​​model 160 is trained via a network of backend servers running instances of the game application. Multiple gameplays 310a to 310n of the game application are shown. As described above, gameplay data 320a to 320n are generated from gameplays 310a to 310n. In one embodiment, multiple gameplays are manipulated by the AI ​​model 160 during training. In this manner, input state data is generated by the analyzer 140 using data from the AI ​​model 160, and the input state data is provided by the analyzer 140 as previous actions during training. The gameplay data may include metadata, such as game state data describing the state of the game at a specific point, controller input data, and recordings of gameplay 310a to 310n for the purpose of extracting metadata and / or training state data. The capture engine 340 captures the gameplay data 320a to 320n as described above, as well as other data that may be provided, such as success criteria 330. The success criteria can be used to distinguish similar training state data for training purposes. In other words, the success criteria may be used by the deep learning engine 190 to train the AI ​​model 160. The training state data 345 is provided to the deep learning engine 190. The functions of the deep learning engine 190 have been described in relation to Figures 3B-1 and 3B-2, but the same applies to the deep learning engine 190 shown in Figure 5C. For simplicity, not all components of the deep learning engine 190 are shown in Figure 5C. Generally, the deep learning engine 190 includes a modeler 120 configured to train and / or build an AI model 160 using training state data 345 and success criteria 330. The modeler 120 can implement artificial intelligence through various neural networks (e.g., convolutional, recurrent, etc.). Specifically, the modeler 120 is given a set of inputs (e.g., features that can define the context or situation (game state) of a game application) and identifies a set of feature-dependent rules that make predictions and / or determine actions to take. For example, the output of the AI ​​model 160 may predict success in progressing through a game application scenario (e.g., defining the gameplay situation of a game application). The set of rules connecting features and / or nodes constitutes the AI ​​model 160.

[0106] As illustrated, the deep learning engine functions in both the learning and application phases. Specifically, gameplay 310 is automatically executed by the AI ​​model 160, etc. In this way, the AI ​​model 160 is self-trained. Specifically, the analyzer 140 includes an input control sequence parser 571, a player profiler 144a, and a map / route tracker 572, which are as previously described. For example, the input control sequence parser 147a is configured to determine the sequence of controller inputs used by the player to control gameplay. The sequence map and / or route tracker 148a of the analyzer 140 is configured to track the progress of gameplay, including tracking progress through the game environment. The player profiler 144a of the analyzer 140 is configured to profile the player playing the game application (for example, to determine the player's skill level).

[0107] Furthermore, the action generator 170 of the analyzer 140 includes an autoplayer 143a configured to autoplay a game application, for example, to automatically train an AI model, as instructed by the AI ​​model trainer 143b, according to a predefined objective. For example, the predefined objective may be to automatically train the AI ​​model 160. Specifically, the analyzer 140 includes an autoplayer 143a configured to autoplay a game application, as instructed by the AI ​​model trainer 143b. For example, with respect to a given set of training state data, such as a training instance, the analyzer 140 may analyze the trained output of the AI ​​model 160 to determine the next set of controller inputs to control the corresponding gameplay. Specifically, the combination of the bot selector and the input control selection / prediction engine 537 of the AI ​​model trainer 143b is configured to determine the next set of controller inputs according to the objective. For example, it may be possible to select a bot optimized to complete the game with the greatest success and efficiency, or to explore various permutations of the game application. Depending on the objective, the input control selection / prediction engine 537 is configured to predict what the next set of controller inputs will be, given the objective and the selected bot. The controller inputs are returned to the autoplayer 143a, which then sends the controller inputs to the corresponding instance of the game application that supports the corresponding gameplay. In this way, training state data 345 can be generated quickly and efficiently by utilizing a bank of backend servers for running instances of the game application for the corresponding gameplays 310a-310n.

[0108] In one embodiment, a set of input controls or controller inputs may be determined to be played following a first input state data. The first set of input controls is selected because it is predicted to satisfy success criteria and is predicted to have the highest success rate among the set of input controls. In this way, the first set of input controls can be delivered as an action to the corresponding instance of the game application for execution.

[0109] Figure 5D shows a data flow diagram illustrating the process of automatically navigating within a game application using an autosweeper / player 146a directed by an AI model 160 trained over a network of backend servers running an instance of the game application, according to one embodiment of the present disclosure. It shows multiple gameplays 501 (e.g., 501a-501n) of the game application. The gameplay can be automatically controlled, for example, via the AI ​​model 160. In this case, the AI ​​model 160 may be configured to provide the following input state data (e.g., controller inputs used to generate input state data such as game states). An instance of the game application may be running on a backend server of a streaming game service (e.g., a cloud gaming system). Gameplay 501a-501n may occur, for example, through one or more scenarios S-1, S-2, and S-3. Input state data 505a-505n from gameplay 501a-501n are provided to an analyzer 140 configured to analyze the output of the AI ​​model 160. The deep learning engine 190 implements the AI ​​model 160 (in the application phase, not the training phase). The AI ​​model 160 is configured to provide an output, and the analyzer 140 is configured to perform or provide an action to be performed based on its analysis of the output.

[0110] Specifically, gameplay 310 is performed automatically by an AI model 160, for example. For example, a predefined objective may be to automatically test a game application after the AI ​​model has been trained. For example, the AI ​​model may have been previously self-trained and then used to test the game application. In another embodiment, an auto-sweep function is performed during training. Specifically, analyzer 140 includes an input control sequence parser 571 and a map / route tracker 572, which are as previously described. For example, the input control sequence parser 147a is configured to determine a sequence of controller inputs that have been previously tested. The sequence map and / or route tracker 148a of analyzer 140 is configured to track the progress of gameplay under test and to track progress in the game environment.

[0111] In one embodiment, first input state data is generated by the AI ​​model as a previous action taken during training. While analyzing the output of the AI ​​model, various permutations for responding to the first input state data are then determined. For example, each permutation includes a unique set of controller inputs to be acquired. One or more actions are then taken, such as executing different permutations. In this way, the system can be configured to detect anomalies when playing a game application.

[0112] Specifically, the action generator 170 of the analyzer 140 includes an autosweeper engine / player 146a configured to explore the game application by using a permutation engine 146b, etc., to determine various permutations that the game application can execute. For example, the input-controlled permutation engine 146b is configured to determine various permutations that the game application can execute in response to a given situation (e.g., a particular game state). That is, the input-controlled permutation engine 146b can determine what the next set of controller inputs should be for a given set of input state data 405a~405n (e.g., a game state). In this case, the permutation engine 146b discovers different permutations to respond to the input state data, and each permutation contains a different set of actions to be taken. The autosweeper engine 146a is configured to go through various permutations by controlling gameplay 401a~401n (e.g., by passing appropriate controller inputs to the execution instance of the game application).

[0113] Furthermore, the analyzer 140 is configured to perform quality analysis of the game application, for example, to discover weak points (e.g., excessively long and tedious sequences, difficult sections, etc.) or defects (e.g., glitches, loops, etc.) in the game application. For example, the map / route analyzer 441 is configured to discover weak points in the game application by analyzing the output of different permutations of the game application (e.g., game states). In one embodiment, the game code identifier 443 is configured to discover problems in the coding of the game application, with code locations 447 provided as output.

[0114] Figure 5E shows a data flow diagram illustrating a process for providing an opponent to a player according to one embodiment of the present disclosure, the opponent being directed by an AI model trained over a network of backend servers running instances of the game application. It shows gameplay 501x ​​of the game application. Gameplay may be controlled by a player Px via a corresponding client device, with instances of the game application running on backend servers of a streaming game service (e.g., a cloud gaming system) as described above. In other embodiments, the game application may run locally on the client device, and metadata is delivered to the backend servers for AI model support. Gameplay 501x ​​occurs during scenario S-1 and is controlled by player input operations 503x.

[0115] Input state data 505x from gameplay 501x ​​is provided to analyzer 140 configured to analyze the output of the trained AI model 160, which is implemented via deep learning engine 190 (in the application phase, not the learning phase). The input state data 505x is received after the AI ​​model 160 has been trained and therefore may not be part of the training state data used to train the AI ​​model described above. The AI ​​model 160 is configured to provide an output, and analyzer 140 is configured to perform or provide an action to be taken based on its analysis of the output.

[0116] Specifically, analyzer 140 includes an input control sequence parser 571 and a player profiler 144a, which are as previously described. For example, the input control sequence parser 147a is configured to determine the sequence of controller inputs used by the player to control gameplay. The player profiler 144a of analyzer 140 is configured to profile the player playing the game application (for example, to determine the player's skill level).

[0117] Furthermore, the analyzer 140 includes a bot builder 575 configured to construct one or more bots (automatic player robots or opponents used in gameplay or to control characters during gameplay). For example, the bot builder 575 may be configured to construct an ultimate bot 142d learned by the AI ​​model 160 by applying success criteria. As described above, for a given set of inputs (e.g., input training data), more successful patterns (e.g., rules including associated features and / or labels) are learned and selected over unsuccessful patterns. In this way, the best or ultimate bot 142d that is most successful in playing the game application is trained. In another example, the bot builder 575 is configured to construct a virtual player or virtual me bot 142c that simulates a first player. In one embodiment, training state data is obtained from gameplay by the first player on one or more client devices. That is, data from the gameplay of other players is not used to train the virtual me bot 142c. In this way, the AI ​​model is trained to memorize metadata created by the first player's gameplay, so the AI ​​model accurately reflects the first player.

[0118] In yet another example, Bot Builder 575 is configured to build one or more bots of varying skill levels. For example, bots of varying skill levels may include Expert Bot 576, Intermediate Bot 577, and Beginner Bot 578. In one embodiment, training state data may be parsed to reflect the corresponding skill level so that the AI ​​model is trained using data of the corresponding skill level. For example, success criteria may be defined so that only gameplay from expert players is used to train the AI ​​model so that Bot Builder 575 can build Expert Bot 576. In another example, success criteria may be defined so that only gameplay from intermediate-skill players is used to train the AI ​​model so that Bot Builder 575 can build Intermediate Bot 577. In yet another example, success criteria may be defined so that only the gameplay of players with beginner skills is used to train the AI ​​model, enabling bot builder 575 to build beginner bot 578. In yet another embodiment, a bot with a specific skill level can be implemented by using a virtual ultimate bot 142d and applying one or more situations to the ultimate bot's performance, including introducing randomness and / or latency. For example, the ultimate bot's performance may be impaired by introducing delays between executions of controller inputs in a sequence, or by introducing random controller inputs into a particular sequence of controller inputs known to be highly likely to succeed in performing the task.

[0119] Furthermore, the action generator 170 of the analyzer 140 includes an autoplayer 143a configured to autoplay a game application, for example, to automatically train an AI model, as instructed by the AI ​​model trainer 143b, according to a predefined objective. For example, the predefined objective may be to automatically train an AI model 160. Specifically, the analyzer 140 includes an autoplayer 143a configured to autoplay a game application, as instructed by the AI ​​model trainer 143b.

[0120] The analyzer 140 includes an autoplayer 143a configured to autoplay the game application for the purpose of providing bot opponents (e.g., automatically reacting robot opponents). The bot opponent selector 142a is configured to select an appropriate bot, such as the bots previously introduced (e.g., Ultimate Bot 142d, Virtual Me Bot 142c, and bots of various skill levels, including Expert Bot 576, Intermediate Bot 577, or Beginner Bot 578). The player controlling the gameplay can control a character that faces off against the bot opponents. Specifically, the autoplayer 143a is configured to implement the selected automated robot (bot). Given a set of input state data 505x, the output can be analyzed by the analyzer 140 via the autoplayer 143a to determine the next set of instructions for controlling the bot found in the game application.

[0121] Furthermore, the bot slotter 142b is configured to apply difficulty settings (for example, to the bot's operation) that are reflected in the corresponding bot. The bot slotter can start with the Ultimate Bot 142d or any other trained bot. Difficulty settings may be applied proactively by the player or based on the user profile. For example, if the player indicated in the profile is an expert, the difficulty will be set high (making it difficult to defeat the opposing bot). Conversely, if the player is a beginner, the difficulty will be set low (making it easy to defeat the opposing bot). As illustrated, the bot throttle 142b may include a randomizer 551 configured to introduce random instructions. A high-difficulty corresponding bot has low-level random instructions inserted into the stream of instructions normally used to control the ultimate bot 142d. As a result, the resulting bot resembles the ultimate bot 142d. On the other hand, a low-difficulty bot will have high-level random instructions inserted into the stream of instructions used to successfully control the ultimate bot 142d. Due to the random instructions, the resulting bot's operation will be unstable and it will not perform as well as the ultimate bot 142d. Furthermore, a latency engine 553 may be implemented to apply difficulty settings. For the resulting high-difficulty bots, the latency introduced into the command stream, which is normally applied to the ultimate bot 142d, will be limited or nonexistent. On the other hand, for the resulting low-difficulty bots (e.g., beginner-friendly), a larger latency will be introduced into the command stream, which is normally applied to the ultimate bot 142d. In this way, the resulting low-difficulty bots will move very slowly during attacks or defensive maneuvers, and will be easily defeated.

[0122] Figure 5F shows a data flowchart illustrating a process according to one embodiment of the present disclosure that provides various services for identifying player weaknesses and training the player to overcome those weaknesses. It shows gameplay 501x ​​of the game application. Gameplay may be controlled by player Px via a corresponding client device, and as described above, an instance of the game application is running on a backend server of a streaming game service (e.g., a cloud gaming system). In other embodiments, the game application may run locally on the client device, and metadata is delivered to the backend server for AI model support. Gameplay 501x ​​occurs during scenario S-1 and is controlled by player input operations 503x.

[0123] Input state data 505x from gameplay 501x ​​is provided to analyzer 140 configured to analyze the output of the trained AI model 160, which is implemented via deep learning engine 190 (in the application phase, not the learning phase). The input state data 505x is received after the AI ​​model 160 has been trained and therefore may not be part of the training state data used to train the AI ​​model described above. The AI ​​model 160 is configured to provide an output, and analyzer 140 is configured to perform or provide an action to be taken based on its analysis of the output.

[0124] Furthermore, the action generator 170 of the analyzer 140 includes an autoplayer 143a that is configured to autoplay a game application, for example, to automatically train an AI model, as instructed by the AI ​​model trainer 143b, according to a predefined objective. For example, the predefined objective may be to automatically train the AI ​​model 160.

[0125] Specifically, the analyzer 140 includes a weakness identifyr 141a configured to determine the weaknesses of a corresponding player controlling corresponding gameplay according to a predefined objective and to provide guidance. The weaknesses are determined by analyzing the player's gameplay.

[0126] For example, the analysis may include a comparison with a success criterion. Specifically, a weakness can be identified by determining that the output from the AI ​​model of the first input state data is lower than the average success rate, based on an analysis of the output compared to other outputs obtained from comparable input state data. For example, for a given input state data (e.g., the first input state data), a first set of interconnected nodes that generate an output in response to the first input state data may have a value lower than the average value determined for satisfying the corresponding success criterion for similar gameplay with comparable input state data. In other words, a comparison is made with gameplay that has the same or similar input state data. Thus, the weakness identifyr 141a can determine the player's weaknesses.

[0127] Furthermore, the weakness trainer 141b is configured to provide services that are useful for the player to overcome their weaknesses. For example, the weakness trainer 141b may provide one or more tutorials 561 (e.g., videos, game sessions, etc.) aimed at improving the player's skills in relation to the player's weaknesses. The tutorials may be video tutorials explaining how to strengthen the player's skills related to the weakness, or game sessions aimed at strengthening skills related to the weakness. Furthermore, training sessions addressing identified weaknesses may be presented to the player via a corresponding client device. In one embodiment, the weakness trainer 141b may also be configured to provide a bot specifically designed to direct the player to play in a way that exposes their weaknesses. The bot may be a virtual ultimate opponent 565 for the player to compete against. The virtual ultimate opponent 565 may be the ultimate bot 142d as described above.

[0128] Figure 6 shows components of an exemplary device 600 that can be used to perform various embodiments of the present disclosure. For example, Figure 6 shows an exemplary hardware system suitable for training an AI model capable of performing a game application and / or various functionalities related to the gameplay of the game application, according to one embodiment of the present disclosure. This block diagram shows a device 600 that can incorporate a personal computer, a server computer, a game console, a mobile device, or other digital device, each of which is suitable for practicing embodiments of the present invention. The device 600 includes a central processing unit (CPU) 602 for running a software application and, optionally, an operating system. The CPU 602 may consist of one or more homogeneous or heterogeneous processing cores.

[0129] According to various embodiments, the CPU 602 is one or more general-purpose microprocessors having one or more processing cores. Further embodiments can be implemented using one or more CPUs having a microprocessor architecture specifically adapted for highly parallel and computationally intensive applications such as media and interactive entertainment applications, among applications configured for deep learning, content classification, and user classification. For example, the CPU 602 may be configured to include an AI engine (e.g., deep learning) 190 configured to support and / or perform learning operations with respect to providing various functionalities (e.g., predictions) in relation to a game application and / or gameplay of a game application. The deep learning engine may include a modeler 120 configured to build and / or train an AI model configured to provide various functionalities related to the game application and / or the gameplay of the game application. Furthermore, the CPU 602 includes an analyzer 140 configured to implement the trained AI model. The trained AI model provides an output in response to an input, and the output depends on the predefined functionalities of the trained AI model. The trained AI model can be used to determine what actions can be taken during gameplay. The analyzer 140 determines which is the appropriate action to take. In other words, the analyzer 140 is configured to perform various functionalities related to the game application and / or the gameplay of the game application. The analyzer 140 is configured to analyze the output from the trained AI model 160 to a given input (e.g., controller input, game state data, success criteria) and provide a response.

[0130] Memory 604 stores applications and data used by the CPU 602. Storage 606 provides non-volatile storage and other computer-readable media for applications and data, and may include fixed disk drives, removable disk drives, flash memory devices, and CD-ROMs, DVD-ROMs, Blu-ray®, HD-DVDs, or other optical storage devices, as well as signal transmission and storage media. User input devices 608 transmit user input from one or more users to device 600, examples of which may be keyboards, mice, joysticks, touchpads, touchscreens, still or video recorders / cameras, and / or microphones. The network interface 614 enables device 600 to communicate with other computer systems via an electronic communication network, which may include wired or wireless communication via a wide area network such as a local area network or the Internet. The audio processor 612 is adapted to generate analog or digital audio output from instructions and / or data provided by the CPU 602, memory 604, and / or storage 606. The components of device 600, including the CPU 602, memory 604, data storage 606, user input device 608, network interface 610, and audio processor 612, are connected via one or more data buses 622.

[0131] The graphics subsystem 614 is further connected to the data bus 622 and the components of device 600. The graphics subsystem 614 includes a graphics processing unit (GPU) 616 and graphics memory 618. The graphics memory 618 includes display memory (e.g., a frame buffer) used to store pixel data for each pixel of the output image. The graphics memory 618 may be integrated into the same device as the GPU 616, connected as a separate device from the GPU 616, and / or incorporated within memory 604. Pixel data can be provided directly from the CPU 602 to the graphics memory 618. Alternatively, the CPU 602 provides the GPU 616 with data and / or instructions defining a desired output image, from which the GPU 616 generates pixel data for one or more output images. The data and / or instructions defining the desired output image can be stored in memory 604 and / or graphics memory 618. In an embodiment, the GPU 616 includes a 3D rendering function for generating pixel data for an output image from instructions and data defining the scene geometry, lighting, shading, texturing, motion, and / or camera parameters. The GPU 616 may further include one or more programmable execution units capable of executing shader programs. In one embodiment, the GPU 616 may be implemented within an AI engine 190 to provide additional processing power, such as for AI or deep learning functions.

[0132] The graphics subsystem 614 periodically outputs pixel data from the graphics memory 618 for an image to be displayed on the display device 610 or for an image to be projected by the projection system 640. The display device 610 may be any device capable of displaying visual information in response to signals from device 600, including CRT, LCD, plasma, and OLED displays. Device 600 can, for example, provide analog or digital signals to the display device 610.

[0133] Accordingly, this disclosure describes systems and methods for implementing deep learning (also known as machine learning) techniques to build an AI model using training data collected from a network of servers running instances of a gaming application that supports one or more gameplays, and to use the trained AI model to provide various functionalities related to the game application and / or the gameplay of the game application.

[0134] It should be understood that the various embodiments defined herein may be combined or assembled into specific embodiments that utilize the various features disclosed herein. Therefore, the provided embodiments are only a few possible embodiments and are not limited to the various embodiments that can be defined by combining different elements. In some embodiments, some embodiments may include fewer elements without departing from the spirit of the disclosed or equivalent embodiments.

[0135] Embodiments of the Disclosure may be implemented in a variety of computer system configurations, including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, miniature computers, and mainframe computers. Embodiments of the Disclosure may also be implemented in distributed computing environments in which tasks are performed by remote processing devices linked via wired or wireless networks.

[0136] With the embodiments described above in mind, it should be understood that embodiments of this disclosure may utilize various computer operations involving data stored in a computer system. These operations require the physical manipulation of physical quantities. Any of the operations described herein that form part of embodiments of this disclosure are useful mechanical operations. Embodiments of the disclosure also relate to devices or apparatus for performing these operations. The apparatus may be built specifically for a required purpose, or it may be a general-purpose computer selectively started or configured by a computer program stored in the computer. Specifically, various general-purpose machines may be used with computer programs written according to the teachings herein, or it may be more convenient to build a more specialized apparatus to perform the required operations.

[0137] The disclosure can also be embodied as computer-readable code on a computer-readable medium. A computer-readable medium is any data storage device that can store data and that can subsequently be read by a computer system. Examples of computer-readable mediums include hard drives, network-attached storage (NAS), read-only memory, random-access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. A computer-readable medium may also include computer-readable tangible media distributed on a network-connected computer system so that the computer-readable code is stored and executed in a distributed manner.

[0138] Although the method operations were described in a specific order, it should be understood that, as long as the processing of the overlay operations is performed in the desired manner, other maintenance operations may be performed between operations, or operations may be coordinated to occur at slightly different times, or operations may be distributed throughout the system to allow processing operations to occur at various processing-related intervals.

[0139] While the foregoing disclosure has been described in some detail for clarity, it will be apparent that certain changes and modifications can be made within the scope of the appended claims. Therefore, these embodiments should be considered illustrative rather than restrictive, and the embodiments of this disclosure are not limited to the details provided herein and may be modified within the scope of the appended claims and equivalents.

Claims

1. A method for processing artificial intelligence (AI) models for game applications, The AI ​​model is trained from the gameplay of the scenario using training state data collected from multiple gameplays of the scenario and the associated success criteria for each of the multiple gameplays. During the first gameplay of the aforementioned scenario, first input state data is received, The first input state data is applied to the AI ​​model to generate an output indicating the success rate of the scenario in the first gameplay. Based on a predefined objective, the analysis of the output is performed. A method for performing actions to achieve the predefined objectives based on the analyzed output.

2. For training the aforementioned AI model, Multiple instances of the game application that support the multiple gameplays are run on multiple servers. The training state data related to the corresponding gameplay of the aforementioned scenario is collected by the multiple servers. Define the aforementioned success criteria, The method according to claim 1, comprising providing the training state data and the success criteria to a deep learning engine.

3. The method according to claim 2, wherein the multiple gameplays are operated by multiple players via multiple client devices.

4. The aforementioned predefined objective is to provide game support. If it is determined that the first input state data does not satisfy the success criteria based on the analyzed output, Based on the analyzed output and the first input state data, the first user profile of the first player operating the first gameplay is determined. The method according to claim 1, wherein the client device of the first player is provided with a recommendation as an action regarding how to play the scenario, the recommendation reflecting the skills of the first player based on the first user profile.

5. In performing the analysis of the output, The aforementioned predefined objective is to provide instruction. By determining that the success rate of the first input state data is lower than average, the weakness of the first player controlling the first gameplay is identified. The method according to claim 1, wherein a training session corresponding to the aforementioned weakness is executed as the action and delivered to the client device of the first player.

6. The aforementioned predefined objective is to provide equivalence to gameplay, After training the AI ​​model, a plurality of input state data are received during a second plurality of gameplays of the scenario, the second plurality of gameplays are operated by a plurality of players, and the plurality of input state data include a plurality of player characteristic metrics of the plurality of players. The plurality of input state data are applied to the AI ​​model to generate a plurality of outputs indicating the success rates of the plurality of second gameplays in the scenario. The multiple outputs are analyzed, and multiple player profiles are determined based on the multiple player characteristic metrics. In order to achieve the aforementioned predefined objectives, a balanced team of players is constructed from the aforementioned multiple players based on their corresponding player profiles. The method according to claim 1, further comprising:

7. Player characteristics metrics include: The accuracy of the compatible player, or, The speed at which the corresponding player generates the controller input sequence, or The reaction time of the corresponding player when an event occurs in the aforementioned scenario, or The consistency of the corresponding player, or The method according to claim 6, wherein the transition time between the first target and the second target of the corresponding player is included.

8. In the execution of the aforementioned action, The aforementioned predefined objective is to automatically train the AI ​​model. The aforementioned multiple gameplays are controlled by the AI ​​model during training. The first input state data is generated by the AI ​​model as a previous action adopted during training. Following the first input state data, the analysis determines a set of multiple controller inputs necessary for the first gameplay to be played. Select a first set of controller inputs that is predicted to meet the aforementioned success criteria and is predicted to have the highest success rate among the set of multiple controller inputs. The method according to claim 2, wherein the set of first controller inputs is distributed as an action to a corresponding instance of the game application and executed.

9. In the execution of the aforementioned action, The aforementioned predefined objective is to automatically test the game application, The first input state data is generated by the AI ​​model as a previous action during training. In response to the first input state data, different permutations are determined as the analysis, and each permutation includes a unique set of controller inputs to be acquired. The method according to claim 2, wherein the different permutations described above are performed as the action to detect an anomaly in the gameplay of the game application.

10. The multiple gameplays of the aforementioned game application are controlled by a first player via one or more client devices. The method according to claim 2, wherein the trained AI model is a virtual player that simulates the first player.

11. A computer-readable medium for storing computer programs for artificial intelligence (AI) training, A program instruction for training the AI ​​model from the multiple gameplays of the scenario in the game application, using training state data collected from multiple gameplays of the scenario and the associated success criteria for each of the multiple gameplays. During the first gameplay of the aforementioned scenario, a program instruction for receiving first input state data is provided, A program instruction for applying the first input state data to the AI ​​model to generate an output indicating the success rate of the scenario in the first gameplay, A program instruction for performing the analysis of the output based on a predefined objective, A computer-readable medium on which program instructions for performing actions to achieve the predefined objectives based on the analyzed output are recorded.

12. The program instructions for training the AI ​​model include: Program instructions for running multiple instances of the game application that support the multiple gameplays on multiple servers, Program instructions for collecting the training state data related to the corresponding gameplay of the aforementioned scenario on the multiple servers, Program instructions for defining the aforementioned success criteria, The computer-readable medium according to claim 11, comprising program instructions for providing the training state data and the success criteria to a deep learning engine.

13. The computer-readable medium according to claim 12, wherein the multiple gameplays are operated by multiple players via multiple client devices.

14. The aforementioned predefined objective is to provide game support. A program instruction for determining that the first input state data does not satisfy the success criteria based on the analyzed output, A program instruction for determining the first user profile of the first player who operates the first gameplay, based on the analyzed output and the first input state data, A computer-readable medium according to claim 11, further comprising providing a recommendation to the client device of the first player as an action, the recommendation reflecting the skills of the first player based on the first user profile, the program instructions for providing the recommendation.

15. The program instructions for performing the analysis of the output are: The aforementioned predefined purpose is to provide instruction, A program instruction for identifying weaknesses of the first player operating the first gameplay by determining that the success rate of the first input state data is lower than average, The computer-readable medium according to claim 11, comprising a program instruction for executing a training session corresponding to the aforementioned weakness as the action and delivering it to the client device of the first player.

16. A computer system, Processor and A memory coupled to the processor, which, when executed by the computer system, stores instructions in the memory that cause the computer system to execute an artificial intelligence (AI) training method; The method comprises, Using training state data collected from multiple gameplays of a scenario, and the associated success criteria for each of the multiple gameplays, an AI model is trained from the multiple gameplays of the scenario in the game application. During the first gameplay of the aforementioned scenario, first input state data is received, The first input state data is applied to the AI ​​model to generate an output indicating the success rate of the scenario in the first gameplay. Based on a predefined objective, the analysis of the output is performed. A computer system that performs actions to achieve the predefined objectives based on the analyzed output.

17. In the above method, the training of the AI ​​model is performed as follows: Multiple instances of the game application that support the multiple gameplays are run on multiple servers. The training state data related to the corresponding gameplay of the aforementioned scenario is collected by the multiple servers. Define the aforementioned success criteria, The computer system according to claim 16, further comprising providing the training state data and the success criteria to a deep learning engine.

18. In the aforementioned method, The computer system according to claim 17, wherein the aforementioned multiple gameplays are operated by multiple players via multiple client devices.

19. In the aforementioned method, The aforementioned predefined objective is to provide game support. If it is determined that the first input state data does not satisfy the success criteria based on the analyzed output, Based on the analyzed output and the first input state data, the first user profile of the first player operating the first gameplay is determined. The computer system according to claim 16, which provides the client device of the first player with a recommendation as an action regarding how to play the scenario, wherein the recommendation reflects the skills of the first player based on the first user profile.

20. In the method described above, when performing the analysis of the output, The aforementioned predefined objective is to provide instruction. By determining that the success rate of the first input state data is lower than average, the weakness of the first player controlling the first gameplay is identified. The computer system according to claim 16, which executes a training session corresponding to the aforementioned weakness as the action and delivers it to the client device of the first player.

Images