Game play operation learning device

Abstract

A gameplay learning device 600 comprises: an acquisition means 621 that acquires play data and labels, the play data including a first play state for a game and an action taken by a player in the first play state, and the labels indicating whether something is an object of learning; a learning means 622 that generates, on the basis of the play data and the labels, a game player model for outputting an action treated as the object of learning with respect to the input of a second play state; and an output means 623 that outputs the game player model.

Description

[Technical Field]

[0001] The present invention relates to a game play operation learning device, a game play operation learning method, and a recording medium. [Background technology]

[0002] 2. Description of the Related Art In various games, such as board games like Go and Shogi, and computer games like fighting games and shooting games, a computer may control characters and the like.

[0003] One example of a technology used for such computer-based control is Patent Document 1. Patent Document 1 describes a fighting game learning device that includes a memory unit for storing various programs and data, and a control unit that controls the movements of multiple characters appearing in the fighting game based on the operation state of an input operation unit and the program stored in the memory unit. According to Patent Document 1, the control unit collects operation data related to the techniques used by the characters in response to operation of the input operation unit and screen state data related to the screen display at predetermined intervals, and executes a learning program to write the collected screen state data at predetermined intervals to a learning data storage unit. The control unit then optimizes the weights of the learning results by performing deep learning calculations based on the screen state data stored in the learning data storage unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-195512 [Non-patent literature]

[0005] [Non-Patent Document 1] Lu Wang, 8 others, “Adversarial Cooperative Imitation Learning for Dynamic Treatment Regimes”, [online], April 20 - 24, 2020, Proceedings of The Web Conference 2020, [Retrieved on September 8, 2021], Internet <URL:https: / / dl.acm.org / doi / 10.1145 / 3366423.<3380248>

Summary of the Invention

Problems to be Solved by the Invention

[0006] In order to perform more appropriate learning, such as being more human-like or more learning-target-like, it is desirable to perform imitation learning as described in Patent Document 1 rather than reinforcement learning in which the learner learns through its own actions. However, in order to perform imitation learning appropriately, a large amount of history such as play data is required. However, when simply collecting play data according to the operations of the player as described in Patent Document 1, it was difficult to sufficiently collect the play data required for imitation learning. As a result, there has been a problem that it is difficult to perform learning to approach the learning target, such as it being difficult to learn the operations of a computer player that are closer to human operations.

[0007] Therefore, an object of the present invention is to provide a game play operation learning device, a game play operation learning method, and a recording medium that can solve the problem that there may be a difficulty in performing learning to approach the learning target.

Means for Solving the Problems

[0008] A game play operation learning device according to one aspect of the present disclosure for achieving such an object is an acquisition means for acquiring play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether it is a learning target; a learning means for generating a game player model for outputting an action of the learning target in response to an input of a second play state, based on the play data and the label; an output means for outputting the game player model; Equipped with The structure is as follows.

[0009] In addition, a game play operation learning method according to another aspect of the present disclosure includes: The information processing device acquiring play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether or not the play state is a learning target; A game player model is generated based on the play data and the label to output the behavior of the learning target in response to the input of a second play state. The structure is as follows.

[0010] Furthermore, a recording medium according to another aspect of the present disclosure includes: In the information processing device, acquiring play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether or not the play state is a learning target; A game player model is generated based on the play data and the label to output the behavior of the learning target in response to the input of a second play state. It is a computer-readable recording medium that records a program for implementing the processing. [Effects of the Invention]

[0011] According to the above-described configurations, it is possible to provide a learning device, a learning method, and a recording medium that enable suitable learning so that the operation of a computer player is closer to the operation of a human, and so that the learning object is closer to the operation of the computer player. [Brief explanation of the drawings]

[0012] [Figure 1]FIG. 2 is a diagram for explaining a learning device according to the first embodiment of the present disclosure. [Figure 2] FIG. 2 is a block diagram illustrating an example of the configuration of a learning device. [Figure 3] FIG. 3 is a diagram showing an example of input data shown in FIG. 2. [Figure 4] FIG. 10 is a diagram illustrating an example of an attribute. [Figure 5] FIG. 10 is a diagram showing an example of play data to be collected. [Figure 6] FIG. 10 is a diagram showing another example of play data. [Figure 7] FIG. 10 is a diagram illustrating another example of play data. [Figure 8] FIG. 10 is a diagram illustrating an example of a learning process. [Figure 9] 10 is a flowchart illustrating an example of the operation of the learning device. [Figure 10] FIG. 10 is a block diagram showing another example configuration of the learning device. [Figure 11] FIG. 10 is a diagram illustrating an example of audio information. [Figure 12] FIG. 10 is a diagram illustrating a configuration example of a learning system according to a second embodiment of the present disclosure. [Figure 13] FIG. 11 is a block diagram showing an example of the configuration of the customer terminal shown in FIG. [Figure 14] FIG. 11 is a block diagram showing an example of the configuration of the server device shown in FIG. [Figure 15] FIG. 10 is a diagram illustrating an example of a billing process. [Figure 16] FIG. 10 is a diagram illustrating another example of the billing process. [Figure 17] 10 is a flowchart illustrating an example of the operation of the server device. [Figure 18] 10 is a flowchart illustrating an example of the operation of the server device. [Figure 19] FIG. 11 is a block diagram showing an example of the hardware configuration of a game play operation learning device according to a third embodiment of the present disclosure. [Figure 20] FIG. 1 is a block diagram illustrating an example of the configuration of a game play operation learning device. [Figure 21] FIG. 10 is a block diagram showing an example configuration of a game player model utilization and provision device according to a fourth embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] [First embodiment] A first embodiment of the present disclosure will be described with reference to FIGS. 1 to 11. FIG. 1 is a diagram illustrating a learning device 100. FIG. 2 is a block diagram illustrating an example configuration of the learning device 100. FIG. 3 is a diagram illustrating an example of input data 121 shown in FIG. 2. FIG. 4 is a diagram illustrating an example of attributes. FIGS. 5 and 6 are diagrams illustrating an example of play data to be collected. FIG. 7 is a diagram illustrating another example of play data. FIG. 8 is a diagram illustrating an example of a learning process. FIG. 9 is a flowchart illustrating an example operation of the learning device 100. FIG. 10 is a block diagram illustrating another example configuration of the learning device 100. FIG. 11 is a diagram illustrating an example of audio information.

[0014] In a first embodiment of the present disclosure, a learning device 100 (game play operation learning device) that performs machine learning based on play data of various games, such as board games like Go and Shogi, and computer games like fighting games and shooting games, is described. As shown in FIG. 1 , the learning device 100 of this embodiment generates a game player model for outputting the behavior of a target game player in response to a play state input, based on play data to which a label indicating whether or not the target game player is a target game player is assigned. That is, the learning device 100 performs machine learning using both play data to which a label indicating the target game player is assigned and play data to which a label indicating the target game player is assigned is not a target game player. Specifically, for example, a first label indicating a success case is assigned to play data having an attribute to be learned, and a second label indicating a failure case, different from the first label, is assigned to play data having an attribute different from the target game player. The learning device 100 then performs machine learning so as to approach play data having the attribute to be learned and to move away from play data having an attribute different from the target game player.

[0015] Learning device 100 is an information processing device that performs machine learning based on game play data acquired from an external device or the like. Games may include board games such as Go and Shogi, computer games such as fighting games and shooting games, and any other games. For example, learning device 100 is a server device. Learning device 100 may be a single information processing device, or may be implemented on the cloud, for example.

[0016] Fig. 2 shows an example configuration of learning device 100. Referring to Fig. 2, learning device 100 has, as main components, for example, a communication I / F unit 110, a storage unit 120, and an arithmetic processing unit 130.

[0017] The communication I / F unit 110 is composed of a data communication circuit, etc. The communication I / F unit 110 performs data communication with an external device, etc., connected via a communication line.

[0018] The storage unit 120 is a storage device such as a hard disk or memory. The storage unit 120 stores processing information and a program 123 required for various processes in the arithmetic processing unit 130. The program 123 is read into the arithmetic processing unit 130 and executed to realize various processing units. The program 123 is read in advance from an external device or recording medium via a data input / output function such as the communication I / F unit 110, and is stored in the storage unit 120. Main information stored in the storage unit 120 includes, for example, input data 121 and a neural network 122.

[0019] The input data 121 includes play data indicating actions taken by the player in the game, the state of the game, etc. The input data 121 is acquired for learning purposes from an external device or the like via the communication I / F unit 110 or the like.

[0020] 3 shows an example of input data 121. As shown in FIG. 3, the input data 121 includes play data with a predetermined attribute that has been labeled as a success case, and play data with an attribute different from the predetermined attribute that has been labeled as a failure case. For example, the input data 121 includes multiple pieces of play data labeled as a success case and multiple pieces of play data labeled as a failure case.

[0021] Here, the attribute indicates information specific to a player, such as the type and skill level of the player playing the game, or the characteristics of the player. FIG. 4 shows an example of the attribute. Referring to FIG. 4, the attribute may include, for example, a player attribute, a skill level attribute, a person attribute, and a specific person attribute. Specifically, for example, the player attribute indicates the type of player, such as whether the player is human or AI (artificial intelligence). The skill level attribute indicates the player's skill in the game, such as advanced player, intermediate player, beginner, xx-dan, or professional. The person attribute indicates information specific to the player, such as address and gender. The specific person attribute indicates a specific person or individual, such as professional A or YouTuber B. The specific person attribute may be, for example, an identifier uniquely assigned to an individual.

[0022] The play data has attributes corresponding to the characteristics of the player playing the game, such as those exemplified above. Note that the attributes may be based on the characteristics of the play data, such as the frequency of a particular action within a given period of time, instead of or in addition to the player's characteristics.

[0023] Furthermore, a label indicating whether or not the play data is a learning target is assigned to the play data in advance, for example, in an external device. Specifically, for example, a first label, a label of success, is assigned to play data having an attribute to be learned, and a second label, a label of failure, different from the first label, is assigned to play data having an attribute different from the attribute to be learned. A failure label may be assigned to play data having an attribute opposite to the attribute to be learned, rather than simply a different attribute.

[0024] For example, when a label of a success example is assigned to play data of an advanced player, a label of a failure example is assigned to play data having attributes different from those of the advanced player, such as an intermediate player or a beginner. Furthermore, when a label of a success example is assigned to play data of an advanced player, a label of a failure example may be assigned to play data having attributes opposite to those of the advanced player, such as a beginner. Furthermore, when a label of a success example is assigned to play data having a specific person attribute indicating a specific person, such as a professional A, a label of a failure example may be assigned to play data that does not have the specific person attribute indicating a specific person. Note that in each of the above examples, a label of a failure example may be assigned to play data having a player attribute indicating that the player is a non-human AI. By assigning a label of a failure example to play data having a player attribute indicating that the player is an AI, it is possible to perform machine learning processing that is more AI-like, i.e., that avoids non-human behavior. For example, by labeling play data having a specific person attribute indicating a specific person as a success example, and labeling play data having a player attribute indicating an AI as a failure example, it becomes possible to update the weight values ​​of the neural network 122 so that the play data approaches that of a specific person and moves away from unhuman behavior.

[0025] In this embodiment, the attributes to be learned may be identified by any means. Alternatively, instead of assigning labels in advance, the learning device 100 may be configured to assign labels to the play data based on information indicating the attributes acquired together with the play data or information indicating the attributes to be learned.

[0026] As described above, the play data included in the input data 121 indicates actions taken by the player in the game, the state of the game, etc. For example, the play data includes state information indicating the state of the game (first play state) in the game, action information indicating the actions taken by the player in the state, etc.

[0027] As an example, FIG. 5 shows an example of play data when a fighting game is played. Referring to FIG. 5, the play data includes information for each target, which is a character competing in a battle. The status information includes at least one of identification information for identifying the character competing in the battle, character information indicating the character's remaining stamina, position information indicating the character's position coordinates and orientation information, and movement information indicating the character's movement speed and movement information indicating the character's movement during action. The movement information includes key information indicating the key pressed by the player. The play data may include information other than the above examples.

[0028] As another example, FIG. 6 shows an example of play data when a game of shogi is played. Referring to FIG. 6, the play data includes information for two players playing shogi. The status information includes at least one of piece position information indicating the position of the pieces, held piece information indicating the type of held pieces, and remaining time information indicating the remaining time. The action information includes at least one of piece type information indicating the type of moved piece, previous position information indicating the position of the moved piece before it was moved, next position information indicating the position of the moved piece, and consumed time information indicating the time consumed before the piece was moved. The play data may include information other than the above examples.

[0029] Thus, the input data 121 includes play data corresponding to the game to be learned. The input data 121 may include play data for each individual scene individually, or may include play data as time-series data in which states and actions are linked together, as shown in FIG. 7. For example, the play data may include a first play state in the game, an action in the first play state, and a third play state resulting from the action. By using time-series data as the learning target, it is possible to adjust weights and the like to enable more appropriate output.

[0030] When play data including state information is input, neural network 122 is subjected to machine learning processing using input data 121, which is training data, so as to output action information corresponding to the state information, etc. In other words, neural network 122 is subjected to machine learning processing so as to output an action to be learned in response to an input of a second play state.

[0031] The arithmetic processing unit 130 has an arithmetic device such as a CPU (Central Processing Unit) and its peripheral circuits. The arithmetic processing unit 130 reads and executes a program 123 from the storage unit 120, thereby realizing various processing units by causing the above hardware and the program 123 to work together. Main processing units realized by the arithmetic processing unit 130 include, for example, an acquisition unit 131, a learning unit 132, and an output unit 133.

[0032] The acquisition unit 131 acquires play data and the like from an external device, etc. For example, the acquisition unit 131 acquires information indicating attributes of the play data along with the play data, etc. The acquisition unit 131 also stores the acquired play data and the like in the storage unit 120 as input data 121.

[0033] Furthermore, the acquisition unit 131 can acquire information indicating attributes to be learned, etc. For example, the acquisition unit 131 may acquire information indicating attributes to be learned together with play data, or may acquire information indicating attributes to be learned at a timing different from that of the play data.

[0034] The learning unit 132 performs machine learning to output a learning target action in response to an input of a second play state, based on play data including a first play state and an action, and input data 121 including a label. For example, the learning unit 132 inputs the input data 121, which is training data, to the neural network 122. The learning unit 132 then updates the weight values ​​of the neural network 122 so that the input data approaches play data labeled as a success case and moves away from play data labeled as a failure case. For example, the learning unit 132 repeats the above process using a large amount of training data to generate a game player model, which is a pre-created model corresponding to the attributes to be learned. Note that the learning unit 132 may perform machine learning processing using known means.

[0035] As an example, as shown in FIG. 8, the learning unit 132 may train an AI that imitates success cases, such as by updating weight values, while having the AI ​​that distinguishes successful actual cases compete with an AI that distinguishes successful actual cases and cooperate with an AI that distinguishes unsuccessful actual cases. Here, the AI ​​that imitates successful cases may be adjusted by, for example, imitation learning based on play data labeled as successful cases included in the input data 121. The AI ​​that distinguishes successful actual cases may be adjusted by machine learning to distinguish successful cases based on play data labeled as successful cases included in the input data 121. The AI ​​that distinguishes unsuccessful actual cases may be adjusted by machine learning to distinguish unsuccessful cases based on play data labeled as unsuccessful cases included in the input data 121. As shown in FIG. 8, the learning unit 132 may adjust each AI by providing feedback to each AI based on the results of having the AI ​​that distinguishes successful cases and the AI ​​that distinguishes unsuccessful cases distinguish play data generated by the AI ​​that imitates successful cases. For example, as described above, the learning unit 132 may be configured to perform machine learning based on the input data 121, which is training data, by performing imitation learning using an adversarial and collaborative generation method that utilizes a neural network. Specifically, for example, the learning unit 132 may perform machine learning processing using a method such as that described in Non-Patent Document 1. Note that the learning method used by the learning unit 132 is not limited to the above-mentioned example. The learning unit 132 may perform machine learning based on the input data 121 using a known method other than the above-mentioned example.

[0036] The learning unit 132 may also be configured to generate training data by labeling play data based on information indicating the attributes to be learned acquired by the acquisition unit 131. For example, the learning unit 132 may label play data having the attributes to be learned as a success case, while labeling play data having attributes different from the attributes to be learned as a failure case. The learning unit 132 may label play data having attributes different from the attributes to be learned that are opposite to the attributes to be learned as a failure case. Note that which attributes are opposite to each other may be determined in advance, for example, or may be determined by the learning unit 132 by any means.

[0037] The output unit 133 outputs a game player model or the like that is a result of learning by the learning unit 132. For example, the output unit 133 can output the game player model or the like to an external device or the like via the communication I / F unit 110 or the like.

[0038] The above is an example of the configuration of learning device 100. Next, an example of the operation of learning device 100 will be described with reference to FIG.

[0039] 9 shows an example of the operation of the learning device 100. Referring to FIG. 9, the acquisition unit 131 The acquisition unit 131 acquires play data and the like from an external device or the like (step S101). The acquisition unit 131 also stores the acquired play data and the like in the storage unit 120 as input data 121.

[0040] The learning unit 132 inputs the input data 121, which is training data, into the neural network 122. Then, the learning unit 132 updates the weight values ​​of the neural network 122 so that the input data 121 approaches play data labeled as a success case and moves away from play data labeled as a failure case. For example, the learning unit 132 performs machine learning processing based on the input data 121 as described above (step S102). Note that the processing of step S101 and the processing of step S102 do not necessarily have to be consecutive.

[0041] As described above, the learning device 100 includes a learning unit 132. With this configuration, the learning unit 132 can perform machine learning processing based on input data 121, which includes play data having a specific attribute to be learned and play data having an attribute different from the play data to be learned. In other words, the learning unit 132 can perform machine learning processing using both play data labeled as a learning target and play data labeled as a non-learning target. As a result, machine learning can be performed based on a larger amount of play data than when machine learning is simply performed based on play data having a specific attribute to be learned. This allows for appropriate learning to approximate the learning target, even when it is difficult to collect a sufficient amount of play data having a specific attribute.

[0042] The configuration of the learning device 100 is not limited to the example shown in Fig. 2. For example, Fig. 10 shows another example configuration of the learning device 100. Referring to Fig. 10, the arithmetic processing unit 130 of the learning device 100 can implement a speech information acquisition unit 134 in addition to the configuration shown in Fig. 2 by executing the program 123.

[0043] The audio information acquisition unit 134 acquires audio information indicating the audio of a specific person. Then, the audio information acquisition unit 134 stores the acquired audio information in the storage unit 120 as audio information 124. For example, when the acquisition unit 131 acquires play data, the audio information acquisition unit 134 acquires information indicating audio having the same specific person attributes as the play data. When information indicating attributes to be learned is acquired at a different timing from the play data, the audio information acquisition unit 134 may acquire the information indicating audio at the same timing as acquiring the information indicating attributes to be learned.

[0044] Fig. 11 shows an example of the voice information 124. Referring to Fig. 11, in the voice information 124, output situation information indicating the situation in which voice is output is associated with voice data for each attribute such as a specific person attribute. For example, in Fig. 11, the voice data "Please take your time" is associated with the situation of "thinking long."

[0045] If the storage unit 120 includes the audio information 124, the output unit 133 can output the audio information 124 corresponding to the learning target together with the game player model, etc., which is the result of learning by the learning unit 132. This allows an external device, etc. that receives the audio information 124, to utilize the results of learning by the learning unit 132 and output audio based on the audio information 124. As a result, it becomes possible to provide an external device, etc. with a communication experience that makes it seem as if the user is playing with a player imitated by AI, for example.

[0046] [Second embodiment] Next, a second embodiment of the present invention will be described with reference to Fig. 12 to Fig. 18. Fig. 12 is a diagram showing an example of the configuration of a learning system 200. Fig. 13 is a block diagram showing an example of the configuration of a customer terminal 300. Fig. 14 is a block diagram showing an example of the configuration of a server device 400. Figs. 15 and 16 are diagrams for explaining an example of a billing process. Figs. 17 and 18 are flowcharts showing an example of the operation of the server device.

[0047] In a second embodiment of the present disclosure, a learning system 200 including a learning device 500 having similar functions to the learning device 100 described in the first embodiment will be described. As will be described later, in this embodiment, the learning device 500 performs machine learning using a method similar to that of the learning device 100 described in the first embodiment, so as to approximate the play data of a specific person, such as a professional e-sports player, a YouTuber, or a celebrity.

[0048] Figure 12 shows an example configuration of a learning system 200. Referring to Figure 12, the learning system 200 has a plurality of customer terminals 300, a server device 400 which is a game player model utilization / providing device, and a learning device 500. As shown in Figure 12, the customer terminals 300 and the server device 400 are connected to each other so as to be able to communicate with each other via a network or the like. Furthermore, the server device 400 and the learning device 500 are connected to each other so as to be able to communicate with each other via a network or the like.

[0049] The customer terminal 300 is an information processing device on which a player plays a game. For example, the customer terminal 300 may be any information processing device such as a video game device that executes a video game, a personal computer, or a tablet terminal.

[0050] Fig. 13 shows the configuration of a customer terminal 300 characteristic of this embodiment. Referring to Fig. 13, the customer terminal 300 includes a play data acquisition unit 310, a transmission unit 320, and a usage instruction unit 330, in addition to components necessary for executing a game. For example, the customer terminal 300 includes a calculation unit such as a CPU and a storage device. For example, the customer terminal 300 can realize each of the above processing units by having the calculation unit execute a program stored in the storage device.

[0051] The play data acquisition unit 310 acquires play data indicating actions taken by the player in the game and the state of the game when the player plays the game. The play data acquisition unit 310 may acquire play data at predetermined intervals, or may acquire play data when a predetermined condition is met, such as when the player performs an action. The play data acquisition unit 310 may also acquire play data as time-series data in which states and actions are linked together. The play data acquired by the play data acquisition unit 310 may be stored in a storage device included in the customer terminal 300.

[0052] The transmitting unit 320 transmits the play data acquired by the play data acquiring unit 310 to the server device 400. The transmitting unit 320 may transmit, together with the play data, information indicating player attributes that is stored in advance in the client terminal 300 to the server device 400. For example, the transmitting unit 320 can transmit the play data and the like to the server device 400 at any timing.

[0053] The use instruction unit 330 instructs the server device 400 to make available a game player model or the like according to the learning results corresponding to a specific person attribute that indicates a specific person. In other words, the use instruction unit 330 transmits to the server device 400 a use instruction requesting transmission of model information or the like necessary to make the game player model available at the customer terminal 300. For example, in response to an input from a player operating the customer terminal 300, the use instruction unit 330 instructs the server device 400 to make available a game player model or the like indicated by the input from the player.

[0054] The server device 400 is an information processing device that accumulates play data and game player models. The server device 400 also receives learning instructions and instructs the learning device 500 to perform learning corresponding to specific person attributes that indicate specific people, and transmits game player models or model information for using game player models to the customer terminals 300 in response to instructions from the customer terminals 300. The server device 400 may be a single information processing device, or may be realized, for example, on the cloud.

[0055] Fig. 14 shows an example of the configuration of the server device 400. Referring to Fig. 14, the server device 400 has, as main components, for example, a communication I / F unit 410, a storage unit 420, and an arithmetic processing unit 430.

[0056] The communication I / F unit 410 is composed of a data communication circuit, etc. The communication I / F unit 410 performs data communication with an external device, etc., connected via a communication line.

[0057] The storage unit 420 is a storage device such as a hard disk or memory. The storage unit 420 stores processing information and programs 423 required for various processes in the arithmetic processing unit 430. The programs 423 are read into the arithmetic processing unit 430 and executed to implement various processing units. The programs 423 are read in advance from an external device or recording medium via a data input / output function such as the communication I / F unit 410, and are stored in the storage unit 420. Main information stored in the storage unit 420 includes, for example, play data information 421 and created model information 422. Note that the storage unit 420 may also include information equivalent to the audio information 124 described in the first embodiment.

[0058] The play data information 421 includes play data received from the customer terminal 300. For example, the play data information 421 stores play data and attributes corresponding to the play data in association with each other. The details of the play data and attributes may be the same as those in the first embodiment.

[0059] The created model information 422 includes a game player model, which is a created model created by performing machine learning processing in the learning device 500. For example, the created model information 422 associates a game player model with information indicating the attributes that were the learning target when the game player model was created.

[0060] The arithmetic processing unit 430 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 430 reads and executes the program 423 from the storage unit 420, thereby implementing various processing functions through cooperation between the above hardware and the program 423. Major processing functions implemented by the arithmetic processing unit 430 include, for example, a play data receiving unit 431, a creation instruction transmitting / receiving unit 432, a created model receiving unit 433, a usage instruction accepting unit 434, an output unit 435, and a billing unit 436.

[0061] The play data receiving unit 431 receives play data and information indicating attributes from the client terminal 300. The play data receiving unit 431 also stores the received information in the storage unit 420 as play data information 421.

[0062] The creation instruction transmitting / receiving unit 432 receives an instruction to create a game player model from an external device such as the customer terminal 300. For example, the creation instruction transmitting / receiving unit 432 receives an instruction to create a game player model together with a specific person attribute, which is an attribute to be learned.

[0063] Furthermore, upon receiving the creation instruction, the creation instruction transmitting / receiving unit 432 references the play data information 421 to identify play data having specific player attributes to be learned. The creation instruction transmitting / receiving unit 432 also references the play data information 421 to identify play data to which a failure example label is to be assigned. As described in the first embodiment, the play data to which a failure example label is to be assigned may be play data having attributes opposite to those of the play data to which a success example label is to be assigned. The creation instruction transmitting / receiving unit 432 then transmits the identified play data and an instruction to create a game player model to the learning device 500.

[0064] The labeling of success cases and failure cases may be performed by the creation instruction transmitting / receiving unit 432 or by the learning device 500. The play data may be transmitted to the learning device 500 in advance. In this case, the creation instruction transmitting / receiving unit 432 may omit the process of identifying and transmitting the play data.

[0065] The created model receiving unit 433 receives from the learning device 500 a game player model, which is a created model created in accordance with the creation instruction sent by the creation instruction transmitting / receiving unit 432. In other words, the created model receiving unit 433 receives from the learning device 500 a game player model created based on play data having attributes to be learned and play data having attributes different from those to be learned. For example, the created model receiving unit 433 receives a game player model and information indicating the attributes that were learned when the game player model was created. The created model receiving unit 433 also stores the received various information in the storage unit 420 as created model information 422.

[0066] The usage instruction receiving unit 434 receives a usage instruction from the customer terminal 300.

[0067] When the usage instruction receiving unit 434 receives a usage instruction from the customer terminal 300, the output unit 435 refers to the created model information 422 to identify a game player model corresponding to the usage instruction. The output unit 435 then transmits to the customer terminal 300 model information and the like required to use the identified game player model. That is, the output unit 435 transmits to the customer terminal 300 model information required to use a game player model, which is a created model created based on play data having attributes to be learned and play data having attributes different from those to be learned. The model information may be the game player model itself, or may be permission information or the like that allows the customer terminal 300 to access the server device 400 and use the game player model. The permission information may be limited to a predetermined time period. The output unit 435 may be configured to transmit audio information 124 with matching attributes along with the game player model or to enable the audio information 124 to be used.

[0068] The billing unit 436 performs billing processing for the customer terminal 300 and the like.

[0069] FIG. 15 shows an example of a billing process by the billing unit 436. Referring to FIG. 15, for example, when the billing unit 436 receives a game player model creation instruction from an external device, such as the client terminal 300, which is the target of the creation, the billing unit 436 can request a registration fee from the external device that sent the creation instruction. For example, the creation instruction transmitting / receiving unit 432 can be configured to transmit a game player model creation instruction, etc., to the learning device 500 on the condition that the billing unit 436 receives the registration fee. The registration fee may be, for example, a predetermined amount. Furthermore, when the output unit 435 transmits model information, etc., to the client terminal 300 in response to a usage instruction from the client terminal 300, the billing unit 436 can request a model usage fee from the client terminal 300 that uses the game player model. For example, the output unit 435 can be configured to transmit model information, etc., to the client terminal 300 on the condition that the billing unit 436 receives the model usage fee. The model usage fee may be, for example, a predetermined amount.

[0070] The billing unit 436 can also be configured to pay a model usage fee to an external device, such as the client terminal 300, that sent the instruction to create a game player model, depending on the number of available game player models. For example, the billing unit 436 may be configured to check whether or not the model usage fee has been paid by checking the number of available game player models at predetermined intervals. The model usage fee may be variable, for example, up to a predetermined upper limit, so that the more the number of uses of the game player model, the higher the fee.

[0071] Furthermore, the billing unit 436 can pay a model provision fee to the learning device 500 when the created model receiving unit 433 receives a game player model from the learning device 500 or when the creation instruction transmitting / receiving unit 432 transmits a game player model creation instruction or the like to the learning device 500. The model provision fee may be, for example, a predetermined amount. The billing unit 436 may also pay an additional usage fee to the learning device 500 according to the number of available game player models, the number of game player model creation instructions, or the like. The additional usage fee may be variable, increasing as the number of available game player models, the number of game player model creation instructions, or the like increases.

[0072] 16, the billing unit 436 may be configured to pay a contract fee to an external device such as the customer terminal 300 instead of or in addition to a registration fee, a model usage fee, or the like. For example, the billing unit 436 may be configured to estimate the number of uses of a game player model and selectively use the process illustrated in FIG. 15 or the process illustrated in FIG. 16 depending on the estimation result, the popularity of the person for whom the game player model is to be created, or the like. Specifically, for example, the billing unit 436 may determine to perform the process illustrated in FIG. 16 instead of the process illustrated in FIG. 15 when it is determined that a predetermined condition is met, such as when the estimated number of uses is equal to or greater than a predetermined value or when it is determined that the popularity is equal to or greater than a predetermined value. In other words, the billing unit 436 may be configured not to request a registration fee when the external device that is the sender of the creation instruction meets a predetermined condition, such as when the estimated number of uses is equal to or greater than a predetermined value or when the popularity is equal to or greater than a predetermined value. In other words, the billing unit 436 requests a registration fee only when the external device that is the sender of the creation instruction satisfies a predetermined condition, such as when the estimated number of uses is less than a predetermined value, when the popularity is less than a predetermined value, etc. Note that the popularity may be calculated by any means based on any information, such as the number of subscribers to the channel corresponding to the person to be created, the number of views of the video, activity history information such as awards won in competitions, whether or not the person has a professional contract, the number of articles in which the person to be created appears, and the number of views.

[0073] The above is an example of the configuration of the server device 400. The server device 400 may be connected to a reinforcement learning device that creates AI by performing reinforcement learning in which a learner learns through their own actions. The server device 400 may also be configured to receive play data between AIs received from the reinforcement learning device as play data having the player attribute "AI." In this case, the server device 400 may be configured to always identify play data having the player attribute "AI" as play data to which failure cases are assigned.

[0074] The learning device 500 has a configuration similar to that of the learning device 100 described in the first embodiment. In this embodiment, the learning device 500 mainly performs machine learning to approach play data having specific person attributes. The learning device 500 also performs machine learning to move away from play data labeled as failure cases.

[0075] The above is an example of the configuration of learning system 200. Next, an example of the operation of server device 400 will be described with reference to FIG.

[0076] 17 shows an example of the operation of the server device 400. Referring to Fig. 17, the creation instruction transmitting / receiving unit 432 receives an instruction to create a game player model from an external device such as the customer terminal 300 (step S201). For example, the creation instruction transmitting / receiving unit 432 receives the instruction to create a game player model together with specific person attributes, which are attributes to be learned.

[0077] The creation instruction transmitting / receiving unit 432 references the play data information 421 to identify play data having specific person attributes to be studied. The creation instruction transmitting / receiving unit 432 also references the play data information 421 to identify play data to which a failure case label is to be assigned. The creation instruction transmitting / receiving unit 432 then transmits the identified play data and an instruction to create a game player model to the learning device 500 (step S202). The creation instruction transmitting / receiving unit 432 may be configured to transmit the instruction to create a game player model to the learning device 500 on the condition that the billing unit 436 receives a registration fee.

[0078] The created model receiving unit 433 receives from the learning device 500 a game player model, which is a created model created in accordance with the creation instruction sent by the creation instruction sending / receiving unit 432 (step S203). For example, the created model receiving unit 433 receives a game player model and information indicating the attributes that were learned when the game player model was created. The created model receiving unit 433 also stores the received various pieces of information in the storage unit 420 as created model information 422.

[0079] 18, the usage instruction receiving unit 434 receives a usage instruction from the client terminal 300 (step S301, Yes). Then, the output unit 435 refers to the created model information 422 to identify a game player model corresponding to the instruction. Then, the output unit 435 transmits the identified game player model to the client terminal 300 (step S302). The output unit 435 may be configured to transmit audio information 124 with matching attributes together with the game player model. The output unit 435 may be configured to transmit the game player model to the client terminal 300 on the condition that the charging unit 436 receives a model usage fee.

[0080] In this way, the server device 400 is configured to provide a game player model created based on play data having specific attributes and play data having attributes different from the above attributes. This configuration makes it possible to provide customers with a gaming experience that is closer to a specific individual and more natural movements.

[0081] The configuration of learning system 200 is not limited to the example shown in this embodiment. For example, this embodiment illustrates a case where play data is stored in server device 400. However, play data may be stored in a location other than server device 400, such as learning device 500. In this case, server device 400 may only output model information without acquiring or storing play data. Furthermore, customer terminal 300, server device 400, etc. may have the functionality of learning device 500. In this way, learning system 200 may employ various modified examples that have similar functions as the entire system.

[0082] [Third embodiment] Next, a third embodiment of the present invention will be described with reference to Figures 19 and 20. Figures 19 and 20 show an example of the configuration of a game play operation learning device 600.

[0083] The game play operation learning device 600 is an information processing device that performs machine learning processing based on play data to which a label indicating whether or not it is a learning target has been assigned. Fig. 19 shows an example of the hardware configuration of the game play operation learning device 600. Referring to Fig. 19, the game play operation learning device 600 has, as an example, the following hardware configuration. ·CPU (Central Processing Unit) 601 (computing unit) ROM (Read Only Memory) 602 (storage device) RAM (Random Access Memory) 603 (storage device) Programs 604 loaded into RAM 603 A storage device 605 for storing the programs 604 A drive device 606 that reads and writes from a recording medium 610 outside the information processing device A communication interface 607 for connecting to a communication network 611 outside the information processing device Input / output interface 608 for inputting and outputting data Bus 609 connecting each component

[0084] 20. The CPU 601 acquires and executes the group of programs 604, thereby enabling the game play operation learning device 600 to realize the functions of acquisition means 621, learning means 622, and output means 623 shown in Fig. 20. The group of programs 604 may be stored in advance in, for example, the storage device 605 or the ROM 602, and the CPU 601 loads and executes the group of programs 604 into the RAM 603 or the like as needed. The group of programs 604 may be supplied to the CPU 601 via the communication network 611, or may be stored in advance in the recording medium 610, with the drive device 606 reading out the programs and supplying them to the CPU 601.

[0085] 19 shows an example of the hardware configuration of the game play operation learning device 600. The hardware configuration of the game play operation learning device 600 is not limited to the above. For example, the game play operation learning device 600 may be configured with only a part of the above configuration, such as omitting the drive device 606.

[0086] The acquisition means 621 acquires play data including a first play state in the game and an action taken by the player in the first play state, and a label indicating whether or not the play state is a learning target.

[0087] The learning means 622 generates a game player model for outputting a learning target action in response to an input of a second play state, based on the play data and the label.

[0088] The output means 623 outputs the game player model.

[0089] As described above, the game play operation learning device 600 includes a learning unit 622. With this configuration, the learning unit 622 can generate a game player model for outputting a learning target's behavior in response to input of a second play state based on the play data and the label. That is, the learning unit 622 can perform machine learning processing using both play data labeled as a learning target and play data labeled as not a learning target. As a result, the learning unit 622 can perform machine learning based on a larger amount of play data than when simply performing machine learning based on play data having a specific attribute to be learned. This allows appropriate learning to approximate the learning target even when it is difficult to collect a sufficient amount of play data having a specific attribute.

[0090] The game play operation learning device 600 described above can be realized by installing a predetermined program in an information processing device such as the game play operation learning device 600. Specifically, a program according to another aspect of the present invention is a program for causing an information processing device such as the game play operation learning device 600 to acquire play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether or not the player is a learning target, and to generate a game player model for outputting the action of the learning target in response to an input in a second play state, based on the play data and the label.

[0091] Furthermore, a game play operation learning method executed by an information processing device such as the above-described game play operation learning device 600 is a method in which the information processing device such as the game play operation learning device 600 acquires play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether or not the action is a learning target, and generates a game player model based on the play data and the label for outputting the action of the learning target in response to an input of a second play state.

[0092] Even if the invention is a program having the above-mentioned configuration, or a computer-readable recording medium having the program recorded thereon, or a game play operation learning method, it can achieve the above-mentioned object of the present invention because it has the same functions and effects as the above-mentioned game play operation learning device 600.

[0093] [Fourth embodiment] Next, a fourth embodiment of the present invention will be described with reference to Fig. 21. Fig. 21 shows an example of the configuration of a game player model utilization / providing device 700.

[0094] The game player model utilization providing device 700 can have the same hardware configuration as the game play operation learning device 600 described in the third embodiment. Furthermore, the game player model utilization providing device 700 can realize the functions of the reception means 721 and the output means 722 shown in Fig. 21 by having a CPU acquire and execute a group of programs. Note that the game player model utilization providing device 700 may employ various modified examples, similar to the game play operation learning device 600 described in the third embodiment.

[0095] The receiving means 721 receives a usage instruction from an external device. The usage instruction is an instruction to enable the external device to use a game player model that has learned the behavior of the learning target in a second play state. For example, the game player model is learned in advance based on play data including a first play state in the game and the behavior taken by the player in the first play state, and a label indicating whether or not the game player model is a learning target.

[0096] In response to the usage instruction received by the receiving means 721, the output means 722 outputs model information for using the game player model indicated by the usage instruction.

[0097] As described above, the game player model utilization / providing device 700 includes output means 722. With this configuration, the output means 722 can output a game player model created by machine learning using both play data labeled as a learning target and play data labeled as not a learning target. As a result, it is possible to provide customers with a gaming experience that is closer to a specific individual, attributes, and more natural movements.

[0098] The above-mentioned game player model utilization providing device 700 can be realized by incorporating a predetermined program into an information processing device such as the game player model utilization providing device 700. Specifically, a program as another aspect of the present invention is a program for realizing processing in an information processing device such as the game player model utilization providing device 700 to receive a usage instruction for enabling the information processing device such as the game player model utilization providing device 700 to use a game player model that has learned the behavior of a learning subject in a second playing state based on play data including a first playing state in a game and actions taken by a player in the first playing state, and a label indicating whether or not the player is a learning subject, and to output model information for using the game player model in accordance with the usage instruction.

[0099] Furthermore, a game player model utilization and provision method executed by an information processing device such as the above-mentioned game player model utilization and provision device 700 is a method in which the information processing device such as the game player model utilization and provision device 700 receives a utilization instruction to make available a game player model that has learned the behavior of the learning subject in a second playing state based on play data including a first playing state in a game and actions taken by the player in the first playing state, and a label indicating whether or not the player is a learning subject, and outputs model information for using the game player model in accordance with the utilization instruction.

[0100] Even if the invention is a program having the above-mentioned configuration, or a computer-readable recording medium having the program recorded thereon, or a game player model utilization provision method, it can achieve the above-mentioned object of the present invention because it has the same functions and effects as the above-mentioned game player model utilization provision device 700.

[0101] <Additional Notes> Some or all of the above embodiments can be described as follows: The following provides an overview of the game play operation learning device and game player model utilization / providing device of the present invention. However, the present invention is not limited to the following configurations.

[0102] (Appendix 1) an acquisition means for acquiring play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether or not the play state is a learning target; a learning means for generating a game player model for outputting an action of the learning target in response to an input of a second play state, based on the play data and the label; an output means for outputting the game player model; A game play operation learning device comprising: (Appendix 2) the label is either a first label assigned to play data having an attribute to be learned, or a second label different from the first label assigned to play data having an attribute different from the attribute to be learned, The learning means performs machine learning using the play data to which the first label has been assigned and the play data to which the second label has been assigned. 2. A game play operation learning device as described in appendix 1. (Appendix 3) the label is either a first label assigned to play data having an attribute to be learned, or a second label different from the first label assigned to play data having an attribute opposite to the attribute to be learned, The learning means performs machine learning using the play data to which the first label has been assigned and the play data to which the second label has been assigned. 3. A game play operation learning device according to claim 1 or 2. (Appendix 4) The learning means performs machine learning so as to approach the play data to which the first label is assigned and to move away from the play data to which the second label is assigned. 4. A game play operation learning device according to claim 2 or 3. (Appendix 5) the second label is assigned to play data having an attribute indicating that the player is an artificial intelligence; The learning means performs machine learning using the play data to which the first label has been assigned and the play data to which the second label has been assigned and which has an attribute indicating that the player is an artificial intelligence. 10. A game play operation learning device according to any one of claims 2 to 4. (Appendix 6) The first label is assigned to play data having an attribute indicating that the player is a specific person. 6. A game play operation learning device according to any one of appendices 2 to 5. (Appendix 7) a voice information acquiring means for acquiring voice information indicating a voice of a player; The output means outputs the audio information. 10. A game play operation learning device according to any one of claims 1 to 6. (Appendix 8) The play data includes the first play state, the action taken by the player in the first play state, and a third play state transition resulting from the action. 10. A game play operation learning device according to any one of claims 1 to 7. (Appendix 9) The information processing device acquiring play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether or not the play state is a learning target; A game player model is generated based on the play data and the label to output the behavior of the learning target in response to the input of a second play state. How to learn gameplay controls. (Appendix 10) In the information processing device, acquiring play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether or not the play state is a learning target; A game player model is generated based on the play data and the label to output the behavior of the learning target in response to the input of a second play state. A computer-readable recording medium that records a program for implementing processing. (Appendix 11) a receiving means for receiving a usage instruction to enable use of a game player model that has learned the behavior of the learning subject in a second play state based on play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether the learning subject is a learning subject; an output means for outputting model information for using the game player model in response to the usage instruction; A game player model utilization providing device comprising: (Appendix 12) a charging means for requesting a model usage fee from a device that uses the game player model; 12. The game player model utilization and provision device according to claim 11. (Appendix 13) an instruction means for instructing a learning device to create the game player model in response to the instruction to create the game player model; The billing means, when receiving the instruction to create the game player model, requests a registration fee from the external device that is the sender of the instruction to create the game player model. 13. The game player model utilization and provision device according to claim 12. (Appendix 14) The charging means charges a registration fee when the external device that is the sender of the creation instruction satisfies a predetermined condition. 14. The game player model utilization and provision device according to claim 13. (Appendix 15) The output means further provides audio information indicating the voice of the player. A game player model utilization and provision device according to any one of appendices 11 to 14. (Appendix 16) The output means provides the game player model created in a state in which a first label is assigned to play data having an attribute to be learned, and a second label different from the first label is assigned to play data having an attribute opposite to the attribute to be learned. A game player model utilization and provision device according to any one of appendices 11 to 15. (Appendix 17) The game player model is a model generated in a state in which a first label is assigned to play data having an attribute indicating that the player is a specific person, and a second label is assigned to play data having an attribute indicating that the player is an artificial intelligence. A game player model utilization and provision device according to any one of appendix 11 to appendix 16. (Appendix 18) The game player model is a model generated by machine learning so as to approach play data to which a first label has been assigned and to move away from play data to which a second label has been assigned. A game player model utilization and provision device according to any one of appendix 11 to appendix 17. (Appendix 19) The information processing device receiving a usage instruction to enable use of a game player model that has learned the behavior of the learning subject in a second play state based on play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether the learning subject is a learning subject; In response to the usage instruction, model information for using the game player model is output. Game player model usage provision method. (Appendix 20) In the information processing device, receiving a usage instruction to enable use of a game player model that has learned the behavior of the learning subject in a second play state based on play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether the learning subject is a learning subject; In response to the usage instruction, model information for using the game player model is output. A computer-readable recording medium that records a program for implementing processing.

[0103] Although the present invention has been described above with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0104] 100 Learning Device 110 Communication I / F section 120 Storage section 121 Input Data 122 Neural Networks 123 Programs 124 Audio Information 130 Processing unit 131 Acquisition Department 132 Learning Department 133 Output section 134 Voice information acquisition unit 200 Learning System 300 customer terminals 310 Play Data Acquisition Unit 320 Transmitter 330 Usage Instructions Department 400 Server device 410 Communication I / F section 420 Storage section 421 Play Data Information 422 Created Model Information 423 Program 430 Processing Unit 431 Play data receiver 432 Creation Instruction Transmitter / Receiver 433 Created Model Receiving Unit 434 User Instructions Reception Department 435 Output Section 436 Billing Department 500 Learning Device 600 Game Play Operation Learning Device 601 CPU 602 ROM 603 RAM 604 Programs 605 Storage device 606 Drive unit 607 Communication Interface 608 Input / Output Interface 609 Bus 610 Recording Media 611 Communication Network 621 Acquisition means 622 Learning Tools 623 Output Means 700 Game Player Model Utilization Providing Device 721 Reception Method 722 Output Method

Claims

1. an acquisition means for acquiring play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether or not the play state is a learning target; a learning means for generating a game player model for outputting an action of the learning target in response to an input of a second play state, based on the play data and the label; an output means for outputting the game player model; Equipped with the label is either a first label that is assigned to play data having a specific person attribute that indicates a specific person, or a second label that is different from the first label that is assigned to play data having a player attribute that indicates an artificial intelligence; The learning means performs machine learning so as to approach the play data to which the first label is assigned and to move away from the play data to which the second label is assigned. Game play operation learning device.

2. a voice information acquiring means for acquiring voice information indicating a voice of a player, The output means outputs the audio information. The game play operation learning device according to claim 1 .

3. The play data includes the first play state, the action taken by the player in the first play state, and a third play state resulting from the action. The game play operation learning device according to claim 1 .

4. An information processing device, acquiring play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether or not the play state is a learning target; generating a game player model for outputting the behavior of the learning target in response to an input of a second play state based on the play data and the label; the label is either a first label that is assigned to play data having a specific person attribute that indicates a specific person, or a second label that is different from the first label that is assigned to play data having a player attribute that indicates an artificial intelligence; When generating the game player model, machine learning is performed so as to approach the play data to which the first label is assigned and to move away from the play data to which the second label is assigned. How to learn gameplay controls.

5. In the information processing device, acquiring play data including a first play state in a game and an action taken by a player in the first play state, and a label indicating whether or not the play state is a learning target; generating a game player model for outputting the behavior of the learning target in response to an input of a second play state based on the play data and the label; the label is either a first label that is assigned to play data having a specific person attribute that indicates a specific person, or a second label that is different from the first label that is assigned to play data having a player attribute that indicates an artificial intelligence; When generating the game player model, machine learning is performed so as to approach the play data to which the first label is assigned and to move away from the play data to which the second label is assigned. A program for implementing processing.

Images