Electronic game provision system and electronic game provision program

Abstract

To provide an electronic game provision system and an electronic game provision program that allow a user to play an electronic game with an AI agent that has learned so as to cause an action in compliance with the taste of a user.SOLUTION: An electronic game provision system allows a user to play an electronic game using an AI agent that has learned so as to use a situation of the electronic game as an input and output an action for the situation. The AI agent learns an answer from the user to a question relating to the electronic game to correspond to the user.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an electronic game providing system and an electronic game providing program. [Background technology]

[0002] A technology has been disclosed that uses a player log obtained by actually playing an electronic game to efficiently train an artificial intelligence agent (AI agent) that plays an electronic game using multiple objects (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-95973 A Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional electronic games, AI agents are generally used solely to play against human players, and their applications are therefore limited. [Means for solving the problem]

[0005] One aspect of the present invention is an electronic game provision system that provides an electronic game, wherein it is possible to play the electronic game using an AI agent that has been trained to take an input of the situation of the electronic game and output an action in response to the situation, and the system is characterized in that it uses the user's answers to questions about the electronic game to train the AI ​​agent to become an AI agent associated with the user.

[0006] Another invention of the present invention is an electronic game providing program for providing an electronic game, which enables a computer to play an electronic game using an AI agent that has been trained to input the situation of the electronic game and output an action in response to the situation, and which uses the user's answers to questions about the electronic game to train the AI ​​agent to become an AI agent associated with the user.

[0007] Here, the questions preferably relate to the user's preferences regarding playing electronic games.

[0008] It is also preferable that the question is about an action to be selected by the user for a situation in an electronic game.

[0009] Preferably, the one situation is a situation that the AI ​​agent encounters when playing an electronic game.

[0010] It is also preferable that the one situation is a situation that a human encounters when playing an electronic game.

[0011] It is also preferable that the one situation is a situation in an electronic game selected by a human being.

[0012] It is also preferable to train the AI ​​agent so that the AI ​​agent outputs actions that are in line with the preferences of the user.

[0013] It is also preferable to train the AI ​​agent so that the AI ​​agent outputs an action selected by the user for the one situation. Effect of the Invention

[0014] According to the present invention, it is possible to provide an electronic game providing system and an electronic game providing program that enable electronic games to be played by an AI agent that has been trained to take actions according to the user's preferences.

[0015] Other objects of the embodiments of the present invention will become apparent by referring to the entire specification. [Brief description of the drawings]

[0016] [Figure 1] 1 is a diagram showing a configuration of an electronic game system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a configuration of a server according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing a configuration of a client according to an embodiment of the present invention. [Figure 4] 10 is a flowchart showing a process of an electronic game service according to an embodiment of the present invention. [Diagram 5] FIG. 11 is a diagram showing an example of a play style selection screen in the embodiment of the present invention. [Figure 6] 4 is a flowchart showing a learning process of an AI agent in the embodiment of the present invention. [Figure 7] FIG. 2 is a functional block diagram showing a configuration of a server according to an embodiment of the present invention. [Figure 8] FIG. 2 is a diagram showing a learning process of an AI agent in an embodiment of the present invention. [Figure 9] FIG. 11 is a diagram showing an example of a play style selection screen in the embodiment of the present invention. [Figure 10] FIG. 11 is a diagram showing an example of a selection request screen according to the embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing an example of an action selection screen according to the embodiment of the present invention. [Figure 12] FIG. 11 is a diagram showing an example of a selection request screen according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] [System Configuration] As shown in Fig. 1, an electronic game system 100 according to an embodiment of the present invention includes a server 102 and a client 104. The client 104 may be a single client or a plurality of clients. The server 102 and the client 104 are connected to each other via an information and communication network 106 such as the Internet so as to be able to exchange information with each other.

[0018] The information and communication network 106 is not limited to the Internet, but may be any network capable of communicatively connecting the server 102 and the client 104 to each other. The information and communication network 106 may be, for example, a dedicated line, a public line (telephone line, mobile communication line, etc.), a wired LAN (Local Area Network), a wireless LAN, etc., or may be a combination of the Internet and these.

[0019] As shown in FIG. 2, the server 102 includes a processing unit 10, a storage unit 12, an input unit 14, an output unit 16, and a communication unit 18. The processing unit 10 includes a means for performing arithmetic processing such as a CPU. The processing unit 10 executes an electronic game server program stored in the storage unit 12, thereby realizing a function for providing an electronic game to the client 104 in the electronic game system 100 in this embodiment. The storage unit 12 includes a storage means such as a semiconductor memory or a memory card. The storage unit 12 is connected to the processing unit 10 so as to be accessible, and stores the electronic game server program and information required for the processing. The input unit 14 includes a means for inputting information. The input unit 14 includes, for example, a keyboard, a touch panel, a button, etc. for receiving input from an administrator. The output unit 16 includes a means for outputting a processing result in the server 102, such as a user interface screen (UI) for receiving input information from the administrator. The output unit 16 includes, for example, a display for presenting an image to the administrator. The communication unit 18 includes an interface for communicating information with the client 104 via the information and communication network 106. The communication by the communication unit 18 may be wired or wireless.

[0020] The server 102 accepts access from each user's client 104 playing an electronic game via an information and communication network 106, stores and manages information about each user in a storage device, and provides each user with an electronic game via the information and communication network 106.

[0021] As shown in FIG. 3, the client 104 includes a processing unit 20, a storage unit 22, an input unit 24, an output unit 26, and a communication unit 28. The client 104 is also called a communication terminal. The processing unit 20 includes a means for performing arithmetic processing such as a CPU. The processing unit 20 executes an electronic game client program stored in the storage unit 22 to realize a function as a client terminal in the electronic game system 100 in this embodiment. The storage unit 22 includes a storage means such as a semiconductor memory or a memory card. The storage unit 22 is connected to the processing unit 20 so as to be accessible, and stores the electronic game client program and information required for the processing. The input unit 24 includes a means for inputting information. The input unit 24 includes, for example, a keyboard, a touch panel, a button, a motion sensor, etc., for receiving input from a user. The output unit 26 includes a means for outputting information required for processing in the client 104, such as a screen for receiving input information from a user and a display for displaying an image when playing an electronic game. The communication unit 28 includes an interface for communicating information with the server 102 via the information and communication network 106. The communication by the communication unit 28 may be wired or wireless.

[0022] Various information processing devices capable of executing a client program for providing an electronic game can be applied as the client 104. For example, the client 104 can be a stationary or portable electronic game terminal installed in a game store or the like, a personal computer (PC), a tablet computer, a smartphone, a mobile phone terminal, a PHS (Personal Handy-phone System) terminal, a personal digital assistant (PDA), a multi-function television receiver equipped with an information processing function (a so-called smart TV), or the like.

[0023] [Electronic games overview] The electronic game system 100 provides electronic games in which AI agents can play against each other, AI agents against a human player, and human players against each other.

[0024] In this embodiment, an electronic game will be described using Texas Hold'em Poker as an example. Texas Hold'em Poker is a type of poker, in which each player is dealt two cards and a maximum of five community cards that are common to all players are combined to make a hand, and the strength of each player's hand determines whether they win or lose.

[0025] In Texas Hold'em, the dealer button is assigned to each player participating in the game in turn. The dealer button moves clockwise after each game, changing the dealer's position. The player to the left of the player with the dealer button is called the small blind, and is obligated to bet half the minimum bet amount. The player to the left of the small blind is called the big blind, and is obligated to bet the minimum bet amount for that game. If there are two players participating in a game, the player without the dealer button will be the big blind, and the player with the dealer button will be the small blind.

[0026] The game begins with each player being dealt two cards face down. These two cards are called "hole cards" or "pocket cards". Once the cards have been dealt, betting can begin. The first betting round is called "pre-flop" and takes place clockwise starting with the player to the left of the big blind. The first player cannot "check" (pass without betting) and must "call" (put up an equal bet) or "raise" (increase the bet) if they do not "fold" (fold). The player who posted the small blind (if not "raised") must put up the remaining half of the bet to call.

[0027] If there are two or more players remaining after the pre-flop betting, the dealer will deal three cards face-up on the table. These cards are called the "flop" and are the common cards (community cards) shared by all players.

[0028] Next, a round of betting on the prop begins, starting with the player to the left of the dealer button and going clockwise. After the betting round on the flop is complete, the dealer deals one more community card on the table. This card is called the "turn", and a round of betting on the turn takes place. After the betting round on the turn is complete, the dealer deals the last community card on the table. This card is called the "river", and a round of betting on the river takes place. If there are two or more players remaining when the betting round on the river is complete, a showdown takes place to determine a winner.

[0029] If two or more players remain after the final bet, a showdown will be held. In the showdown, the winner will be determined by the strength of the hand using the two cards in the player's hand and the five strongest cards from the five community cards that have been played.

[0030] [Electronic game provision service] In the electronic game system 100, multiple users can become players and play electronic games using their respective clients 104. In addition to such human players, electronic games can also be played by AI agents associated with users.

[0031] In this embodiment, the processing of the electronic game of Texas Hold'em Poker will be described with reference to the flowchart of FIG. 4. However, the electronic game is not limited to this, and other electronic games may be used as long as they can be played using an AI agent. In the following processing, the bet may be points that can be used in the electronic game.

[0032] In step S10, a user logs in. Through the processing in this step, the electronic game system 100 functions as an initial setting means.

[0033] A user who wishes to play an electronic game inputs login information required for login processing using the input unit 24 of the client 104 used by the user. The login information may be, for example, a user ID and a password. The client 104 transmits the login information to the server 102 via the communication unit 28 and the information and communication network 106. When the server 102 receives the login information via the communication unit 18, the server 102 permits the login if the login information satisfies the conditions for permitting the login. For example, the server permits the login if the combination of the user ID and password included in the login information matches a combination of a user ID and password registered in advance.

[0034] In addition, in electronic games played by multiple players such as Texas Hold'em Poker, a matching process is performed to pair users who will become players. The server 102 pairs users who have logged in as players who will play poker at one table. It is preferable that the player can be selected to be either the human user himself or an AI agent associated with the user.

[0035] In step S12, a display process of a game image is performed. Through the process in this step, the electronic game system 100 functions as a game image display means. When the login process and the matching process are completed, the server 102 transmits information for displaying a game screen to the client 104 via the communication unit 18 and the information communication network 106. When the client 104 receives the information for displaying the game screen via the communication unit 28, it causes the output unit 26 to display the game screen based on the information.

[0036] In the Texas Hold'em Poker of this embodiment, as shown in FIG. 5, a game screen 300 is displayed. The game screen 300 is provided with a game status display area 302 and an action selection area 304. The game status display area 302 displays information indicating the status of the electronic game. In the case of Texas Hold'em Poker, the game status display area 302 includes the status of the players, the status of the common cards (community cards), the status of the cards (pocket cards) dealt to each player, and the betting status (betting actions of each player). The electronic game status may also include the match history and stats of each player. The action selection area 304 displays actions that the player can select for the electronic game status displayed in the game status display area 302.

[0037] In step S14, a card distribution process is performed. Through the process in this step, the electronic game system 100 functions as a card distribution means. The server 102 performs a process of distributing the hands and community cards of the players participating in the electronic game. The server 102 transmits information on the cards distributed through the information communication network 106 via the communication unit 18 to the clients 104 used by each player. Upon receiving the information on the cards distributed, the client 104 causes the output unit 26 to display the information on the cards in the game status display area 302 of the game screen 300.

[0038] In step S16, a betting round is processed. The electronic game system 100 functions as a betting processing means by the processing in this step. If the user is the player, the user checks the electronic game status displayed in the game status display area 302, and then selects an action from the options displayed in the action selection area 304 using the input unit 24 of the client 104. The client 104 transmits information on the selected action to the server 102 via the communication unit 28 and the information communication network 106. If the AI ​​agent is the player on behalf of the user, the electronic game status is input to the AI ​​agent by processing of the AI ​​play processing unit of the server 102, which will be described later, and an action output from the AI ​​agent in response to the input is automatically selected.

[0039] In step S18, it is determined whether or not the dealing of cards has been completed. Through the processing in this step, the electronic game system 100 functions as a dealing determination means. The server 102 determines whether or not the number of cards to be dealt as the hand and community cards of each player have already been dealt. If further card dealing is required, the process returns to step S14, and if no card dealing is required, the process proceeds to step S20.

[0040] In step S20, after the cards have been dealt, a further betting round is processed. The processing in this step causes the electronic game system 100 to function as a betting processing means. If the user is the player, the user checks the electronic game status displayed in the game status display area 302, and then selects an action from the options displayed in the action selection area 304 using the input unit 24 of the client 104. The client 104 transmits information on the selected action to the server 102 via the communication unit 28 and the information communication network 106. If an AI agent is the player on behalf of the user, the electronic game status is input to the AI ​​agent by processing in the server 102, which will be described later, and an action output from the AI ​​agent in response to the input is automatically selected.

[0041] In step S22, a showdown process is performed. The process in this step causes the electronic game system 100 to function as a post-game processing means. When the process of placing a bet is completed, the server 102 performs a showdown and distributes the bet to the players according to the strength of each player's hand. Information on the bet given to each player is stored as a user database or the like in the storage unit 12 of the server 102. In addition, the situation when the electronic game is played in the above process, the action selected for that situation, and the result of the showdown (the result of the payment and receipt of the bet, etc.) may be stored as game log information in the storage unit 12 of the server 102.

[0042] By repeating the above process, electronic game system 100 provides an electronic game service.

[0043] [How AI agents learn] In the electronic game system 100, an electronic game can be played by an AI agent associated with a user who operates a client 104. Hereinafter, a learning method for an AI agent in the electronic game system 100 will be described with reference to the flowchart shown in FIG.

[0044] 7, the processing unit 10 of the server 102 functions as a supervised learning unit 30, an AI play processing unit 32, an AI log collection unit 34, a reinforcement learning unit 36, a play style learning unit 38, a question processing unit 40, an AI selection unit 42, an AI adjustment unit 44, and a user learning unit 46. The memory unit 12 of the server 102 also stores an AI agent 50, supervised learning data 52, an AI log database (AI log DB) 54, play style-specific AI agents 56, a question answering database (question answering DB) 58, and a user-specific AI agent 60. Note that the client 104 may be responsible for some of the processing of the above functions.

[0045] The AI ​​agent 50 is an artificial intelligence program that plays electronic games. The AI ​​agent 50 learns using a method that uses deep learning, which will be described later. The AI ​​agent 50 includes a multi-layer neural network. The multi-layer neural network is defined by various parameters (such as a layer structure, a neuron structure in each layer, and the weight or bias of each neuron).

[0046] In step S30, supervised learning is performed on the AI ​​agent. The process in this step corresponds to the process by the supervised learning unit 30. The supervised learning unit 30 performs a process of learning the AI ​​agent 50 using supervised learning data 52. The supervised learning data 52 is data that defines rewards for actions in response to situations when playing an electronic game.

[0047] The status of playing an electronic game includes, for the current game (round), the number of players participating in Texas Hold'em Poker, the player's game history and stats, the status of cards common to all players (community cards), the status of cards dealt to oneself (pocket cards), and the betting status (betting actions of each player). The status of playing an electronic game may also include, for past games (rounds), the number of players participating in Texas Hold'em Poker, the player's game history and stats, the status of cards common to all players (community cards), the status of cards dealt to oneself (pocket cards), and the betting status (betting actions of each player).

[0048] The reward is set based on the result of the action performed by the player. The reward is set high when the action performed by the player in the situation when playing the electronic game paired in the supervised learning data 52 results in a favorable result in the electronic game, and is set low when an unfavorable result is obtained.

[0049] For example, in Texas Hold'em Poker, the higher the total amount of bets obtained when multiple showdowns are performed, the higher the reward is set. Also, for example, the total amount of bets obtained when multiple showdowns are performed may be compared between players, and the reward may be set higher in the order of the total amount of bets obtained. Also, for example, the reward may be set higher if there is a winner at each showdown, and lower if there is a loser. Also, for example, the higher the amount of bets obtained when there is a winner at each showdown, the higher the reward may be set.

[0050] The supervised learning data 52 can be set from game situations obtained when actually playing Texas Hold'em Poker, actions for the situations, and the results. The supervised learning data 52 may also be set by combining game situations that would occur when a Texas Hold'em Poker expert (professional, etc.) plays Texas Hold'em Poker, and actions that the expert would take for the situations, and setting rewards for the actions. Game theory (GTO) may also be applied to Texas Hold'em Poker, and learning may be performed using actions that are considered to be suitable for the game situations.

[0051] The supervised learning unit 30 trains the AI ​​agent 50 using the supervised learning data 52. That is, as shown in Fig. 8, the AI ​​agent 50 is trained to output actions with higher rewards using as input the situations when playing an electronic game included in the supervised learning data 52.

[0052] In step S32, an electronic game using an AI agent is played. The processing in this step corresponds to processing by the AI ​​play processing unit 32. The AI ​​play processing unit 32 performs processing to make the AI ​​agents 50 (including the play style-specific AI agents 56 and the user-specific AI agents 60 described later) play an electronic game. In this embodiment, a plurality of AI agents 50 are made to play Texas Hold'em Poker as players. The AI ​​agents 50 apply the multi-layer neural network obtained in the learning and various parameters therefor (layer structure, neuron structure in each layer, weight or bias of each neuron, etc.) to make it possible to play an electronic game by performing an action output from the AI ​​agents 50 when the current situation when playing the electronic game is input.

[0053] The AI ​​log collection unit 34 collects information on the play of the electronic game by the AI ​​agent 50 in the AI ​​play processing unit 32 as log information. The log information collected by the AI ​​log collection unit 34 is stored as an AI log DB 54 in the storage unit 12 of the server 102. The log information includes the status of the electronic game when the electronic game is played by the AI ​​agent 50, the action taken in response to the status, and the result. When the electronic game is Texas Hold'em Poker, the status of the electronic game includes the number of AI agents 50 participating as players, the match history and stats of the AI ​​agents 50, the status of the common cards (community cards), the status of the cards (pocket cards) dealt to the AI ​​agent 50 itself, and the betting status (betting action of each AI agent 50). The status of the electronic game may include not only the current status but also the past status. When the electronic game is Texas Hold'em Poker, the result of the electronic game includes information on whether the AI ​​agent 50 won or lost at the showdown, and the amount of bets won if the AI ​​agent 50 won.

[0054] In step S34, reinforcement learning of the AI ​​agent is performed. The process in this step corresponds to the process by the reinforcement learning unit 36. The reinforcement learning unit 36 ​​performs reinforcement learning, which is learning for strengthening the AI ​​agent 50. Reinforcement learning is a method different from supervised learning, and for example, Q-learning can be used as the reinforcement learning. Specifically, in reinforcement learning, the situation in the electronic game is used as an input, and a reward is given to the AI ​​agent 50 based on the result of the action output by the AI ​​agent 50, and the AI ​​agent 50 learns the action to be output based on the reward.

[0055] In this embodiment, the reinforcement learning unit 36 ​​performs learning based on the results of play of the electronic game by the AI ​​agent 50 under the control of the AI ​​play processing unit 32.

[0056] The reward design in the reinforcement learning of the AI ​​agent 50 by the reinforcement learning unit 36 ​​is not particularly limited. For example, the reward is set higher as the total amount of bets obtained when a showdown is performed multiple times for actions performed on the electronic game situation, using the electronic game situation obtained when playing the electronic game as input data. Then, the AI ​​agent 50 is trained so that when the electronic game situation is input to the AI ​​agent 50, the AI ​​agent 50 outputs an action that will result in a higher reward. In addition, for example, the total amount of bets obtained when a showdown is performed multiple times may be compared between players, and the reward may be set higher in the order of the total amount of bets obtained. In addition, for example, the AI ​​agent 50 may be trained by setting the reward higher when the player is a winner at each showdown and setting the reward lower when the player is a loser. In addition, for example, the AI ​​agent 50 may be trained by setting the reward higher as the player is a winner at each showdown and setting the reward lower.

[0057] The reinforcement learning by the reinforcement learning unit 36 ​​can be performed using log information of the electronic game. The situation of the electronic game contained in the log information is input to the AI ​​agent 50, and a reward is given according to the action output by the AI ​​agent 50 in response to the input, so that learning can be performed. Note that the log information may be log information obtained when an NPC (Non Player Character; a rule-based AI other than the AI ​​agent 50) plays an electronic game, or log information obtained when a human user plays an electronic game as a player.

[0058] In addition, the AI ​​play processing unit 32 may allow the AI ​​agents 50 to play a new electronic game with each other, while providing rewards according to the actions output by the AI ​​agents 50 and the associated results in the electronic game, thereby allowing the AI ​​agents 50 to learn.

[0059] In step S36, a process is performed to generate an AI agent according to a play style. The process in this step corresponds to the process by the play style learning unit 38. After the supervised learning unit 30 and the reinforcement learning unit 36 ​​perform learning for the electronic game to make the AI ​​agent 50 somewhat stronger, the play style learning unit 38 learns the AI ​​agent 50 for each play style to generate an AI agent 56 for each play style. The play style learning unit 38 performs learning so that a plurality of AI agents 56 for each play style are generated according to the play style by various reward designs.

[0060] For example, the electronic game situation when playing an electronic game is used as input data, and a higher reward is set when a bet is obtained after an action corresponding to a bluff in the electronic game situation, that is, an action of continuing to bet even though one's hand is less favorable than the other player's hand. Then, when the electronic game situation is input to the AI ​​agent 50, the AI ​​agent 50 is trained so that an action corresponding to a bluff that can obtain a higher reward is likely to be output from the AI ​​agent 50. In this way, a play style-specific AI agent 56 that is good at bluffing is generated.

[0061] Also, for example, the electronic game situation when playing an electronic game is used as input data, and a higher reward is set when a bet is obtained after a loose action in the electronic game situation, that is, an action of continuing to bet despite a low winning rate of one's own hand at the time of "pre-flop" or "flop". Then, when the electronic game situation is input to the AI ​​agent 50, the AI ​​agent 50 is trained so that it is more likely to output loose actions that can obtain a higher reward. In this way, a play style-specific AI agent 56 with a loose play style is generated.

[0062] Also, for example, the electronic game situation when playing an electronic game is used as input data, and the reward is set higher when a wager is obtained after an aggressive action, i.e., a "bet" or "raise" action, is performed in the electronic game situation. Then, when the electronic game situation is input to the AI ​​agent 50, the AI ​​agent 50 is trained so that an action corresponding to an aggressive action that can obtain a higher reward is easily output from the AI ​​agent 50. In this way, a play style-specific AI agent 56 with an aggressive play style is generated.

[0063] Also, for example, the electronic game situation when playing an electronic game is used as input data, and a lower reward is set when a passive action is taken in the electronic game situation, that is, an action such as "call" or "raise" and then losing a bet. Then, when the electronic game situation is input to the AI ​​agent 50, the AI ​​agent 50 is trained so that an action with a lower reward is less likely to be output from the AI ​​agent 50. In this way, a play-style-specific AI agent 56 with a passive play style is generated.

[0064] As an example of reward design, the higher the total amount of bets obtained when a showdown is performed multiple times for actions performed on the electronic game situation obtained when playing an electronic game, the higher the reward is set. For example, when a player wins a higher bet amount compared to other players when multiple showdowns are performed, the reward is set to +1, and when a player wins only a lower bet amount compared to other players, the reward is set to -1. In this way, the AI ​​agent 50 learns to output actions that can obtain a higher bet amount. Furthermore, when an action that matches each play style is performed at each showdown, a reward is given. For example, when an action that matches the play style is performed, the reward is set to +0.1, and when an action that does not match the play style is performed, the reward is set to -0.1. While adjusting the reward in this way, the AI ​​agent 50 learns to output actions that can obtain the highest possible rewards using the play situation of the electronic game as input, and an AI agent 56 for each play style is generated.

[0065] It should be noted that play styles are not limited to the above-mentioned bluff, loose, aggressive, and passive, and it is possible to similarly train the AI ​​agent 50 for other play styles to generate play style-specific AI agents 56 corresponding to each play style.

[0066] Even if the electronic game is not Texas Hold'em Poker, the AI ​​agent 50 can be trained by setting rewards for actions output from the AI ​​agent 50 according to the play style, thereby generating AI agents 56 for different play styles.

[0067] In step S38, a process of selecting a play-style specific AI agent 56 according to a play style corresponding to the user is performed. The process in this step corresponds to the process by the question processing unit 40 and the AI ​​selection unit 42.

[0068] The question processor 40 presents a question about the electronic game to the user and acquires an answer to the question from the user. The answer acquired from the user is stored as a question and answer DB 58 in the storage unit 12 of the server 102.

[0069] The question about the electronic game is preferably, for example, a question about the user's preference in the electronic game. For example, as shown in FIG. 9, a play style selection screen 200 is displayed on the output unit 26 of the client 104, and a question such as "Which play style is closest to yours?" is presented to the user. In addition, as an answer to the question, the play style selection screen 200 displays options indicating the user's preference in playing the electronic game, and the user is allowed to select the preference in playing the electronic game. For example, as shown in FIG. 9, options such as "I want to play toying with my opponent with bluffs," "I want to play regardless of my hand," "I want to win by attacking aggressively," and "I want to play with solid defense" are displayed, and the user is allowed to select one of the options. The options only need to include content that can confirm the user's preference in the electronic game, and may be stored in advance as the question and answer DB 58 in the storage unit 12 of the server 102. The user uses the input unit 24 of the client 104 to select (tap, etc.) one of the options displayed on the play style selection screen 200.

[0070] Information on the option selected by the user is transmitted from the client 104 to the server 102. The server 102 stores the information on the option received from the client 104 in the storage unit 12 as the question and answer DB 58.

[0071] The AI ​​selection unit 42 refers to the question and answer DB 58 and selects an AI agent 56 for each play style to be used in the electronic game based on the user's answer.

[0072] For example, when an answer "I want to play toying with my opponent by bluffing" is selected on the play style selection screen 200 shown in FIG. 9, a play style-specific AI agent 56 corresponding to bluffing is selected for the user who made that answer. When an answer "I want to play regardless of the hand" is selected, a play style-specific AI agent 56 corresponding to loose is selected for the user who made that answer. When an answer "I want to win by attacking aggressively" is selected, a play style-specific AI agent 56 corresponding to aggressiveness is selected for the user who made that answer. When an answer "I want to play solid defense" is selected, a play style-specific AI agent 56 corresponding to passiveness is selected for the user who made that answer.

[0073] The question processor 40 may also ask a question about an action to be selected by the user for one situation in the electronic game. The question is preferably a question for confirming what action the user will select for a situation in the electronic game in which there are multiple actions that have the same expected value for the effect of the electronic game (for example, the expected value of the bet to be won). In other words, it is preferable to ask a question that reflects the user's play style depending on which action the user selects from multiple actions that have the same expected value for the effect of the electronic game.

[0074] 10, a selection screen 202 is displayed on the output unit 26 of the client 104, and the user is prompted to select whether or not to select an action. The selection screen 202 displays a message saying "I'm going to ask you 50 questions, and I'll watch your actions and learn," and also presents two options, "Learn by Actions," for receiving the user's action selection, and "Skip," for skipping and starting the electronic game immediately, for the user to select one of them. Information on the option selected by the user is transmitted from the client 104 to the server 102.

[0075] When "skip" is selected, the server 102 proceeds to a learning process of the user-specific AI agent 60 by the user learning unit 46, which will be described later. On the other hand, when "learning by action" is selected, the server 102 proceeds to a process of obtaining an action selection from the user.

[0076] When "learning by action" is selected, an action selection screen 204 is displayed on the output unit 26 of the client 104 as shown in FIG. 11. The action selection screen 204 displays "please select an action" to prompt the player to answer the number of questions and the question. The action selection screen 204 also has a game status display area 210 and an action selection area 212. The game status display area 210 displays information indicating the status of the electronic game. When the electronic game is Texas Hold'em Poker, the game status display area 210 includes the players, the status of the common cards (community cards), the status of the cards (pocket cards) dealt to the player, and the betting status (betting actions of each player). The electronic game status may also include the match history and stats of each player. The action selection area 212 displays actions that the player can select for the electronic game status displayed in the game status display area 210. The electronic game status and action options only need to include content that can confirm the user's preferences regarding the electronic game, and may be stored in advance as question and answer DB 58 in storage unit 12 of server 102. After confirming the electronic game status displayed in game status display area 210, the user uses input unit 24 of client 104 to select an action that the user wishes to perform from the action options displayed in action selection area 212.

[0077] Information on the option of the action selected by the user is transmitted from the client 104 to the server 102. The server 102 associates the information on the option received from the client 104 with the status of the electronic game displayed in the game status display area 210, and stores it in the storage unit 12 as the question and answer DB 58.

[0078] The AI ​​selection unit 42 refers to the question and answer DB 58 and selects an AI agent 56 for each play style to be used in the electronic game based on the user's answer.

[0079] For example, a similarity for each play style of the play style-specific AI agent 56 can be set for each action provided as an answer to each question, and the play style-specific AI agent 56 to be used in the electronic game can be selected based on the similarity between the action selected and answered by the user for each question and each play style of the play style-specific AI agent 56. When there are multiple questions, the play style-specific AI agent 56 with the highest similarity can be selected according to the cumulative value or average value of the similarity between each action selected for the multiple questions and each play style. Also, for example, the similarity can be found by calculating the KL distance of the loss distribution between the action selected and answered by the user for each of multiple electronic game situations and each play style of the play style-specific AI agent 56.

[0080] In this embodiment, based on the similarity with the action selected by the user in response to questions about multiple poker playing situations, one of the play style-specific AI agents 56 corresponding to bluffing, the play style-specific AI agent 56 corresponding to loose, the play style-specific AI agent 56 corresponding to aggressive, and the play style-specific AI agent 56 corresponding to passive is selected.

[0081] In addition, the user's selection of a play style on the play style selection screen 200 and the user's selection of an action on the action selection screen 204 may be combined to select a play style-specific AI agent 56 that corresponds to the play style desired by the user.

[0082] In step S40, a process is performed in which the play-style-specific AI agent 56 is adjusted to match the user's skill. The process in this step corresponds to the process performed by the question processing unit 40 and the AI ​​adjustment unit 44.

[0083] The question processor 40 asks a question for the user to evaluate his / her skill in one situation in the electronic game. The question is preferably a question for confirming what action the user will select in a situation in the electronic game where the number of actions that can obtain the effect of the electronic game is narrowed down to one. In other words, it is preferable to set the question as a question that can evaluate the user's skill in the electronic game depending on whether the user selects the correct action, and the action that maximizes the expected value of the favorable effect obtained in the electronic game among a plurality of actions that can be performed in the situation of the electronic game is the correct answer. For example, it is preferable to set the question as a question that can evaluate the user's skill in the electronic game depending on whether the user selects the correct action when the situation in poker is shown by asking a question. Also, for example, a question may be used in which the game theory (GTO) is applied to Texas Hold'em Poker, and an action that is considered to be suitable for the game situation is the correct answer.

[0084] As shown in Fig. 12, a selection screen 206 is displayed on the output unit 26 of the client 104, and the user is prompted to select whether or not to select an action. The selection screen 206 displays a message saying "We will now ask 50 questions and evaluate your skills by watching your actions," and also presents two options, "Evaluate Skills by Actions," for receiving the user's action selection, and "Skip," for skipping and starting the electronic game immediately, for the user to select one of them. Information on the option selected by the user is transmitted from the client 104 to the server 102.

[0085] If "skip" is selected, the server 102 transitions to learning processing of the user-specific AI agent 60 by the user learning unit 46, which will be described later. On the other hand, if "learning by action" is selected, the server 102 transitions to processing for obtaining an action selection from the user. Note that it is also possible to arrange for an evaluation of the user's skill to be performed without making a selection on the selection necessity screen 206. It is also possible for an evaluation of the user's skill to be performed without making a selection on the selection necessity screen 206.

[0086] When "Evaluation of skill by action" is selected, the output unit 26 of the client 104 displays an action selection screen 204 as in FIG. 11. The action selection screen 204 displays "Please select an action" to prompt the user to answer the number of questions and the question. The action selection screen 204 also has a game status display area 210 and an action selection area 212. The game status display area 210 displays information indicating the status of the electronic game. The action selection area 212 displays actions that the player can select for the status of the electronic game displayed in the game status display area 210. The electronic game status and action options may include content that can evaluate the user's skill in the electronic game, and may be stored in advance as the question and answer DB 58 in the storage unit 12 of the server 102. After checking the status of the electronic game displayed in the game status display area 210, the user uses the input unit 24 of the client 104 to select an action that the user wants to perform from the action options displayed in the action selection area 212.

[0087] The AI ​​adjustment unit 44 refers to the question and answer DB 58, and adjusts the play style-specific AI agent 56 selected by the AI ​​selection unit 42 in accordance with the user's answer to the question "evaluation of skill by action" obtained by the question processing unit 40. The AI ​​adjustment unit 44 adjusts the strength of the play style-specific AI agent 56 in the electronic game in accordance with the accuracy rate of the action selected and answered by the user.

[0088] A weak play style specific AI agent 56 is defined as being inferior in some way to a strong play style specific AI agent 56 in accuracy, and the strength of the play style specific AI agent 56 can be adjusted by making adjustments such that the accuracy of inference of the weak play style specific AI agent 56 is hindered compared to the inference of the strong play style specific AI agent 56. Specifically, the strength of the play style specific AI agent 56 can be adjusted by leaving out a portion of information about the play situation of the electronic game and inputting it to the play style specific AI agent 56. For example, in the case of a poker game, the strength of the play style specific AI agent 56 is adjusted by leaving out a portion of information about the hands held by the player himself or herself and other players and inputting it to the play style specific AI agent 56.

[0089] In step S42, a process is performed to train the play-style-specific AI agent 56 to suit the user. The process in this step corresponds to the process by the user learning unit 46. The user learning unit 46 trains the play-style-specific AI agent 56 using information indicating the characteristics of the user's play in the electronic game, in order to make the play-style-specific AI agent 56 closer to the user.

[0090] The user learning unit 46 can use the user's answers to questions about the electronic game to train the AI ​​agent 50 to become the user-specific AI agent 60 associated with the user. For example, when game log information of a past time when the user himself played an electronic game as a player is stored in the storage unit 12, the situation when the electronic game was played, which is included in the log information, is input to the play style-specific AI agent 56, and learning is performed so that an action selected by the user for the situation is output, thereby generating the user-specific AI agent 60. Also, for example, information indicating the situation of the electronic game, which is included in the question asked when grasping the user's play style in step S38, may be input to the play style-specific AI agent 56, and learning is performed so that an action selected by the user for the situation is output, thereby generating the user-specific AI agent 60. Also, for example, information indicating the situation of the electronic game, which is included in the question asked when grasping the user's skill in step S40, may be input to the play style-specific AI agent 56, and learning is performed so that an action selected by the user for the situation is output, thereby generating the user-specific AI agent 60.

[0091] It should be noted that the process of generating the user-specific AI agent 60 by the user learning unit 46 is not limited to this. For example, the user may be asked a question indicating an example of a new electronic game situation, and the action selected by the user in response to the question may be obtained, and the play style-specific AI agent 56 may be trained using this information to generate the user-specific AI agent 60.

[0092] The user can play an electronic game using the user-specific AI agent 60 generated as described above. That is, the user can play an electronic game using the user-specific AI agent 60 corresponding to the user's play style and skill as a proxy for the user. Specifically, the AI ​​play processing unit 32 inputs the play situation of the electronic game to the user-specific AI agent 60, and the user can play the electronic game by applying the actions output from the user-specific AI agent 60.

[0093] In this embodiment, supervised learning in step S30, reinforcement learning in step S34, learning for specific play styles in step S36, adjustment of the AI ​​agent in step S40, and learning tailored to the user in step S42 are all applied, but processing may be performed excluding some of these processes.

[0094] As described above, according to the present embodiment, it is possible to provide an electronic game system, an electronic game program, and an electronic game method that have not been available in the past.

[0095] [Configuration of the invention] [Configuration 1] An electronic game providing system for providing an electronic game, comprising: An electronic game can be played using an AI agent trained to take an electronic game situation as an input and to output an action in response to said situation; 1. A system for providing electronic games, comprising: a computer program for executing an electronic game on a user's behalf; a computer program for executing an electronic game on a user's behalf; [Configuration 2] The electronic game providing system according to configuration 1, The electronic game providing system, wherein the questions relate to the user's preferences regarding playing electronic games. [Configuration 3] The electronic game providing system according to the first or second aspect of the present invention comprises: The electronic game providing system is characterized in that the question is related to an action to be selected by the user for a situation in the electronic game. [Configuration 4] The electronic game providing system according to configuration 3, The electronic game providing system is characterized in that the one situation is a situation encountered by the AI ​​agent when playing the electronic game. [Configuration 5] The electronic game providing system according to configuration 3, The electronic game providing system is characterized in that the one situation is a situation encountered by a human being when playing an electronic game. [Configuration 6] The electronic game providing system according to configuration 3, The electronic game providing system is characterized in that the one situation is a situation selected by a human from among situations in the electronic game. [Configuration 7] The electronic game providing system according to configuration 2, An electronic game providing system, characterized in that the AI ​​agent is trained so that actions output from the AI ​​agent are in line with the preferences of the user. [Configuration 8] The electronic game providing system according to any one of configurations 3 to 6, An electronic game providing system, characterized in that the AI ​​agent is trained so that the AI ​​agent outputs an action selected by the user for the one situation. [Configuration 9] An electronic game providing program for providing an electronic game, comprising: Computer, An electronic game can be played using an AI agent trained to take an electronic game situation as an input and to output an action in response to said situation; 1. A program for providing an electronic game, comprising: a program for providing an electronic game, the program using a user's answers to questions about the electronic game to train the AI ​​agent to become an AI agent associated with the user. [Explanation of symbols]

[0096] 10 Processing unit, 12 Memory unit, 14 Input unit, 16 Output unit, 18 Communication unit, 20 Processing unit, 22 Memory unit, 24 Input unit, 26 Output unit, 28 Communication unit, 30 Supervised learning unit, 32 AI play processing unit, 34 AI log collection unit, 36 Reinforcement learning unit, 38 Play style learning unit, 40 Question processing unit, 42 AI selection unit, 44 AI adjustment unit, 46 User learning unit, 50 AI agent, 52 Learning data, 54 AI log database, 56 AI agent by play style, 60 AI agent by user, 100 Electronic game system, 102 Server, 104 Client, 106 Information and communication network, 200 Play style selection screen, 202 Selection necessity screen, 204 Action selection screen, 206 Selection necessity screen, 210 Game status display area, 212 Action selection area, 300 Game screen, 302 Game status display area, 304 action selection area.

Claims

1. An electronic game providing system for providing an electronic game, comprising: An electronic game can be played using an AI agent trained to take an electronic game situation as input and to output an action in response to said situation; An electronic game providing system, characterized in that the AI ​​agent is trained to become an AI agent associated with the user using the user's answers to questions about the electronic game.

2. 2. The electronic game providing system according to claim 1, The electronic game providing system, wherein the questions relate to the user's preferences regarding playing electronic games.

3. 3. The electronic game providing system according to claim 1, The electronic game providing system is characterized in that the question is related to an action to be selected by the user for a situation in the electronic game.

4. 4. The electronic game providing system according to claim 3, The electronic game providing system is characterized in that the one situation is a situation encountered by the AI ​​agent when playing the electronic game.

5. 4. The electronic game providing system according to claim 3, The electronic game providing system is characterized in that the one situation is a situation encountered by a human being when playing an electronic game.

6. 4. The electronic game providing system according to claim 3, The electronic game providing system is characterized in that the one situation is a situation selected by a human from among situations in the electronic game.

7. 3. The electronic game providing system according to claim 2, An electronic game providing system, characterized in that the AI ​​agent is trained so that actions are output from the AI ​​agent in accordance with the user's preferences.

8. 2. The electronic game providing system according to claim 1, An electronic game providing system, characterized in that the AI ​​agent is trained so that an action selected by the user is output from the AI ​​agent for the one situation.

9. An electronic game providing program for providing an electronic game, comprising: Computer, An electronic game can be played using an AI agent trained to take an electronic game situation as input and to output an action in response to said situation; An electronic game providing program, characterized in that the AI ​​agent is trained to become an AI agent associated with the user using the user's answers to questions about the electronic game.

Images