Model fusion-based card type prediction method, device, equipment and storage medium

Abstract

The application belongs to the technical field of chess and card games, and discloses a card type prediction method, device and equipment based on model fusion and a storage medium. The method comprises the following steps: constructing a game environment; determining a reward strategy according to the game rules and preset requirements; obtaining current card game information, establishing a sample database according to the current card game information, the reward strategy and a reinforcement learning model; training the reinforcement learning model according to the sample database, establishing a game model; obtaining game data according to the game model; training a supervised learning model according to the game data, obtaining a card type prediction model; obtaining prediction data according to target card game information and the card type prediction model, and making a move according to the prediction data. Through the above method, the characteristics of reinforcement learning from nothing to something are combined with the high efficiency of supervised learning to train the final robot model, effectively solve the cold start problem of the chess and card game robot in a new play, and greatly reduce the project operation cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of chess and card games, and particularly relates to a card type prediction method and device based on model fusion, equipment and a storage medium. BACKGROUND

[0002] Thanks to the popularity of machine learning technology, more and more chess and card game products use machine learning technology to develop their robot systems. There are mainly two categories of common robot systems based on machine learning: one is to learn the playing behavior patterns of similar players from massive player game data by using supervised learning; the other is to automatically "explore" a set of playing methods with relatively high level from scratch by using reinforcement learning through self-play. Both methods have advantages and disadvantages: supervised learning needs a large amount of player data as a basis, which is obviously not suitable for a new play that has not yet been launched to players; reinforcement learning can be from scratch and is suitable for new plays, but compared with supervised learning, the amount of computation required in the reasoning process is large, and the robot system using this method cannot be highly loaded, resulting in high project development and maintenance costs.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a card type prediction method and device based on model fusion, equipment and a storage medium, aiming to solve the technical problems that the use of supervised learning in the prior art cannot handle the cold start of new plays in chess and card games, and the use of reinforcement learning has high project development and maintenance costs.

[0005] To achieve the above purpose, the present application provides a card type prediction method based on model fusion, which comprises the following steps:

[0006] Constructing a game environment, the game environment at least comprising game rules and game flow;

[0007] Determining a reward strategy according to the game rules and preset requirements;

[0008] Obtaining current game information, and establishing a sample database according to the current game information, the reward strategy and the reinforcement learning model;

[0009] Training the reinforcement learning model according to the sample database, and establishing a game model;

[0010] Obtaining game data according to the game model and the game environment;

[0011] Training a supervised learning model according to the game data, and obtaining a card type prediction model;

[0012] According to the target game information and the card type prediction model, prediction data is obtained, and the prediction data is used for making a move.

[0013] Optionally, the current game information is obtained, and a sample database is established according to the current game information, a reward strategy and a reinforcement learning model, and the sample database includes:

[0014] The reinforcement learning model is established according to a preset neural network structure.

[0015] The current game is initialized according to the game flow and the game rules, and current game information is obtained.

[0016] The current game information, the reward strategy and the reinforcement learning model are used to determine current move data, and the current move data is stored in a self-game database.

[0017] The current game information is updated, and whether the current game is ended is determined according to the updated current game information.

[0018] When the current game is ended, game result data is obtained, and current sample data is determined according to the game result data and the self-game database.

[0019] The current sample data is stored in a sample database.

[0020] Optionally, the current game information, the reward strategy and the reinforcement learning model are used to determine current move data, and the current move data is stored in a self-game database.

[0021] The current game information is converted according to a preset data format to obtain current game data.

[0022] The current game data is input into the reinforcement learning model to obtain initial move data.

[0023] The initial move data is scored according to the reward strategy to obtain move score data.

[0024] The current move data is determined according to the move score data and the initial move data.

[0025] Optionally, after the current game information is updated and whether the current game is ended is determined according to the updated current game information, the method further includes:

[0026] When the current game is not ended, the updated current game information is returned to the step of inputting the current game information into the reinforcement learning model to obtain initial move data.

[0027] Optionally, the reinforcement learning model is trained according to the sample database to establish a game model, and the game model includes:

[0028] determine a sample quantity according to the sample database;

[0029] train the reinforcement learning model according to sample data in the sample database when the sample quantity meets a preset cumulative quantity, obtain a new reinforcement learning model, and record an iteration number;

[0030] determine a game model according to the new reinforcement learning model when the iteration number is greater than or equal to a preset iteration threshold.

[0031] Optionally, after the step of determining the sample quantity according to the sample database, the method further includes:

[0032] initialize the self-game database when the sample quantity does not meet the preset cumulative quantity, and return to the step of initializing the game information according to the game flow and the game rules to obtain the current game information;

[0033] after the step of training the reinforcement learning model according to the sample data in the sample database when the sample quantity meets the preset cumulative quantity, obtaining the new reinforcement learning model, and recording the iteration number, the method further includes:

[0034] initialize the self-game database when the iteration number is less than the preset iteration threshold, and return to the step of initializing the game information according to the game flow and the game rules to obtain the current game information according to the new reinforcement learning model.

[0035] Optionally, the step of training the supervised learning model according to the game data to obtain the card type prediction model includes:

[0036] establishing a supervised learning model according to a preset neural network structure;

[0037] preprocessing the game data to obtain game matrix data and game label data;

[0038] training the supervised learning model according to the game matrix data and the game label data to obtain the card type prediction model.

[0039] In addition, to achieve the above object, the application further provides a card type prediction device based on model fusion, which comprises:

[0040] an environment construction module configured to construct a game environment, wherein the game environment at least includes game rules and a game flow;

[0041] a reinforcement learning module configured to determine a reward strategy according to the game rules and a preset requirement;

[0042] The reinforcement learning module is further configured to acquire current game information, and establish a sample database according to the current game information, the reward strategy and the reinforcement learning model;

[0043] The reinforcement learning module is further configured to train the reinforcement learning model according to the sample database, and establish a game model;

[0044] The reinforcement learning module is further configured to obtain game data according to the game model;

[0045] The supervised learning module is configured to train a supervised learning model according to the game data, and obtain a card type prediction model;

[0046] The supervised learning module is further configured to obtain prediction data according to target game information and the card type prediction model, and make a card according to the prediction data.

[0047] In addition, to achieve the above object, the application further provides a card type prediction device based on model fusion, which comprises a memory, a processor and a card type prediction program based on model fusion stored in the memory and executable on the processor.

[0048] In addition, to achieve the above object, the application further provides a storage medium, which stores a card type prediction program based on model fusion, and the card type prediction program based on model fusion implements the steps of the card type prediction method based on model fusion when executed by a processor.

[0049] In the present application, by constructing a game environment, determining a reward strategy according to the game rules and preset requirements, obtaining current card game information, establishing a sample database according to the current card game information, the reward strategy and the reinforcement learning model, training the reinforcement learning model according to the sample database, establishing a game model, obtaining game data, training a supervised learning model according to the game data, obtaining a card type prediction model, obtaining prediction data according to the target card game information and the card type prediction model, and making a card according to the prediction data. Compared with the traditional game robot which cannot handle the "cold start" of new play by using the supervised learning method alone, or cannot achieve high bearing by using the reinforcement learning method alone, resulting in high maintenance cost, the present application uses the characteristics of reinforcement learning from nothing to something, first trains a robot model with high level, based on this model, a large number of games are played to generate a large amount of game data; then combined with the efficiency of supervised learning, a card type classifier is trained as the final robot model by using the game data generated by the reinforcement learning model as data samples, the combination of reinforcement learning and supervised learning effectively solves the "cold start" problem of the chess and card game robot in new play, and greatly reduces the project operation cost. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a structural schematic diagram of a card type prediction device based on model fusion of a hardware running environment related to an embodiment scheme of the present application;

[0051] Figure 2 is a flowchart of a first embodiment of a card type prediction method based on model fusion of the present application;

[0052] Figure 3 is a game matrix data schematic diagram of an embodiment of the card type prediction method based on model fusion of the present application;

[0053] Figure 4 is a schematic diagram of the overall flow of an embodiment of the card type prediction method based on model fusion of the present application;

[0054] Figure 5 is a flowchart of a second embodiment of the card type prediction method based on model fusion of the present application;

[0055] Figure 6 is a reinforcement learning flowchart of an embodiment of the card type prediction method based on model fusion of the present application;

[0056] Figure 7 is a structural block diagram of a first embodiment of the card type prediction device based on model fusion of the present application.

[0057] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are merely illustrative of the present application and do not limit the present application in any way.

[0059] Referring to Figure 1 , Figure 1 The hardware environment of the model fusion-based card type prediction device involved in the embodiment of the present application is shown in the structural schematic diagram.

[0060] As Figure 1 shown, the model fusion-based card type prediction device can include a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display screen, an input unit such as a keyboard, and can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a wireless fidelity (Wi-Fi) interface). The memory 1005 can be a high-speed random access memory (RAM) memory, or a stable non-volatile memory (NVM), such as a disk memory. The memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0061] Those skilled in the art can understand Figure 1 that the structure shown in the foregoing is not a limitation on the model fusion-based card type prediction device, and can include more or fewer components than the diagram, or combine certain components, or different component arrangements.

[0062] As Figure 1 shown, the memory 1005 as a storage medium can include an operating system, a network communication module, a user interface module, and a model fusion-based card type prediction program.

[0063] In Figure 1The network interface 1004 shown in the model fusion-based card type prediction device is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the model fusion-based card type prediction device can be arranged in the model fusion-based card type prediction device, the model fusion-based card type prediction device calls the model fusion-based card type prediction program stored in the memory 1005 through the processor 1001, and executes the model fusion-based card type prediction method provided in the embodiment of the application.

[0064] The embodiment of the application provides a model fusion-based card type prediction method, which refers to Figure 2 , Figure 2 The embodiment of the application provides a model fusion-based card type prediction method, which refers to

[0065] In the embodiment, the model fusion-based card type prediction method comprises the following steps:

[0066] Step S10: Constructing a game environment, wherein the game environment at least comprises game rules and a game process.

[0067] It should be noted that the execution subject of the embodiment is a computer, which can be any computer capable of running the model fusion-based card type prediction program, and the embodiment does not limit this. Through the model fusion-based card type prediction program, a chess and card game robot is constructed, and the prediction of the card type in the card game is realized.

[0068] It can be understood that the game in the embodiment is a chess and card game, and the game environment is a complete chess and card game environment, which comprises game rules, a game process and card type judgment, etc. The embodiment does not limit this. The game rules are chess and card game rules, which comprise the determination mode of the first card dealer, all available card types, the card playing order, the size relationship of the card face, etc. The game process is the proceeding process of the chess and card game, which comprises game starting, card dealing, card dealer determination, card playing by the card dealer and game ending, etc. The card type judgment is the judgment of the card type situation, for example, the card face has a spade 10, a heart 10 and a square 3, and it can be judged that the card type has a single card and a pair.

[0069] In the specific implementation, a complete game environment needs to be written for subsequent reinforcement learning, and through simple API (Application Programming Interface, application programming interface) calling, the normal game process can be completed on the console.

[0070] Step S20: Determining a reward strategy according to the game rules and a preset requirement.

[0071] It should be understood that the preset demand is a demand of the chess game robot, for example, high win rate and high score, and the reward strategy is a reward calculation method in reinforcement learning, which can be designed based on win or loss or actual score, for example, winning 1 point and losing 1 point, and the embodiment does not limit this, and the actual situation can be set.

[0072] It should be noted that different reward strategies can obtain different playing behavior patterns, for example, the robot trained based on the win or loss reward strategy can have a high win rate, but can not understand "losing less and winning more", and the robot trained based on the actual score reward strategy can have a lower win rate than the win or loss reward strategy, but can obtain a higher total score after a plurality of games.

[0073] In a specific implementation, the reward calculation method is formulated according to the game rules and the demand of the chess game robot.

[0074] Step S30: obtaining current game information, and establishing a sample database according to the current game information, a reward strategy and a reinforcement learning model.

[0075] It can be understood that the current game information is pre-dealing information, including current hands, the number of remaining cards of each player, the history of cards of each player, a card counter, a bottom card and the like. The reinforcement learning model is an initial model designed, and the parameters of the model are all initialized random parameters, which can be trained through reinforcement learning. The sample database is a database for storing sample data required for training the model.

[0076] In a specific implementation, an initial reinforcement learning model is first designed, the reinforcement learning model is called to predict playing cards through a game environment, and game data generated by self-play is cached to establish a sample database, which is used for training of a subsequent reinforcement learning model.

[0077] Step S40: training the reinforcement learning model according to the sample database to establish a game model.

[0078] Further, the step S40 includes: determining a sample quantity according to the sample database, training the reinforcement learning model according to sample data in the sample database when the sample quantity meets a preset cumulative quantity, obtaining a new reinforcement learning model, and recording an iteration number, and determining a game model according to the new reinforcement learning model when the iteration number is greater than or equal to a preset iteration threshold.

[0079] It should be understood that the sample quantity is the number of sample data in the sample database, the preset cumulative quantity is a set sample quantity threshold, which can be set in the model fusion-based card type prediction program, and the embodiment is set to 32x100=3200, the iteration number is the training number of the reinforcement learning model, and the preset iteration threshold is a set training number threshold, which generally needs to be more than 1 billion times. The game model is a robot model with high level trained by reinforcement learning.

[0080] In a specific implementation, the reinforcement learning model is trained and updated when the sample quantity accumulates to 3200, the game environment can call the latest model in the next card prediction, and the reinforcement learning model is saved after a specified iteration number, and the game model is obtained.

[0081] Step S50: Obtain game data according to the game model and the game environment.

[0082] It should be noted that the game data is a large amount of card game data generated by using the game environment and the game model, and needs to be specific to each card in the card game. The data quantity generally needs to reach the level of tens of millions of times, which can be set in the model fusion-based card type prediction program, and the embodiment does not limit this.

[0083] Step S60: Train the supervised learning model according to the game data to obtain a card type prediction model.

[0084] Further, the step S60 includes: establishing a supervised learning model according to a preset neural network structure, preprocessing the game data to obtain game matrix data and game label data, training the supervised learning model according to the game matrix data and the game label data, and obtaining a card type prediction model.

[0085] It can be understood that the preset neural network structure is the network structure of the model, the embodiment is designed based on a convolutional neural network (CNN), the supervised learning model is an initial model designed, the parameters of the model are all initialized random parameters, the supervised learning model can be trained, and the card type prediction model is a final generated chess and card game robot model. For example Figure 3As shown, the game matrix data is a numerical matrix obtained after encoding each card playing, and the game label data refers to the label corresponding to the card type in each card playing, which can be determined according to the correspondence between the card type and the label, for example: a certain card game has only three card types: single, pair and three, and an encoding is made for the three card types, 1 representing single, 2 representing pair and 3 representing three, and 1, 2 and 3 are labels here. If a card playing in the game data is 3, the game label data is 1 (single 3). As shown in Table 1, the card type category corresponds to the label, which is the correspondence between the card type and the label set in this embodiment.

[0086] Table 1

[0087] Rank category Label Rank category Label No play 1 Three with one 182 Rocket / Queen bomb 1 Three with two 156 Single 15 Single sequence 35 Pair 13 Double sequence 52 Three 13 Three sequence 45 Bomb 13

[0088] In a specific implementation, based on the CNN, a network structure of a supervised learning model is designed, the game data generated by the game model is preprocessed, the game process is converted into game matrix data and game label data, the game matrix data and the game label data are used as sample data to train the supervised learning model, and when the model tends to be stable, a final card type prediction model is obtained, and a corresponding model file is saved and generated.

[0089] It should be noted that the output of the card type prediction model is the probability distribution of each label (card type), for example: the probability distribution of single is 0.1, the probability distribution of pair is 0.2, and the probability distribution of three is 0.7.

[0090] Step S70: obtaining prediction data according to the target game information and the card type prediction model, and making a card playing according to the prediction data.

[0091] It can be understood that the target game information is the situation information that needs to be predicted by using the robot model, and the prediction data is the model output obtained according to the target game information, that is, the probability distribution corresponding to each card type, and the card type with the largest probability distribution is selected as the card playing card type, for example: the output prediction data is "1:0.1, 2:0.2, 3:0.7", and the labels "1, 2 and 3" correspond to single, pair and three respectively, and since the probability distribution of three is the largest, the selected card playing is three.

[0092] In a specific implementation, the target game information is converted into input data in a corresponding format, which is input into the card type prediction model to obtain the probability distribution corresponding to each card type, and the card type with the largest probability distribution is selected as the card playing card type.

[0093] It should be understood that the present embodiment is mainly divided into two parts: reinforcement learning part and supervised learning part. The reinforcement learning part is responsible for constructing the game environment, designing the reward, establishing the game model, and generating the game data by self-play. The supervised learning system is responsible for the pre-processing of the game data, the establishment of the card type prediction model, and the output of the card type prediction model file. As shown in the overall flowchart Figure 4 , the game data generated by the reinforcement learning model is used as a data sample to train the supervised learning model to obtain the card type prediction model as the final chess game robot model, and a model file is generated.

[0094] In the present embodiment, by constructing the game environment, determining the reward strategy according to the game rules and the preset requirements, obtaining the current game information, establishing the sample database according to the current game information, the reward strategy and the reinforcement learning model, training the reinforcement learning model according to the sample database, establishing the game model, obtaining the game data, training the supervised learning model according to the game data, obtaining the card type prediction model, obtaining the prediction data according to the target game information and the card type prediction model, and making a move according to the prediction data. The present embodiment utilizes the characteristics of reinforcement learning from nothing to something, first trains a robot model with high level, based on this model, a large number of games are played to generate a large amount of game data; then, combined with the efficiency of supervised learning, the game data generated by the reinforcement learning model is used as a data sample to train a card type classifier as the final robot model. The combination of reinforcement learning and supervised learning effectively solves the "cold start" problem of the chess game robot in the new play, and greatly reduces the project operation cost.

[0095] Reference Figure 5 , Figure 5 is a flowchart of a second embodiment of a card type prediction method based on model fusion.

[0096] Based on the above first embodiment, the step S30 comprises:

[0097] Step S301: establishing a reinforcement learning model according to a preset neural network structure.

[0098] In specific implementation, the reinforcement learning model is designed, including: determining the data to be input and its format, the design of the model structure, etc. Generally speaking, the model structure based on CNN will be slightly better than the Multilayer Perceptron (MLP).

[0099] Step S302: initializing the current game according to the game flow and the game rules to obtain the current game information.

[0100] It should be noted that the current game is the game being played.

[0101] Step S303: determining current card data according to the current game information, the reward strategy and the reinforcement learning model, and storing the current card data into the self-game database.

[0102] Further, the step S303 comprises: converting the current game information according to a preset data format to obtain current game data, inputting the current game data into the reinforcement learning model to obtain initial card data, scoring the initial card data according to the reward strategy to obtain card scoring data, and determining the current card data according to the card scoring data and the initial card data.

[0103] It can be understood that the preset data format is the format of the data input into the reinforcement learning model, and the embodiment adopts the form of an encoding matrix, which can also be other formats, and the embodiment does not limit the same. The current game data is a numerical matrix obtained by converting the current game information. The initial card data is all legal card types that can be played by the current hand, for example, if the hand is 55556666, the initial card data is 5, 6, 55, 66, 555, 666, 5556, 6665, 55566, 66655, 555666, 5555, 6666, 55556666, the card scoring data is the score corresponding to each card type, for example, if the reward strategy is -0.1 for a single card, -0.1 for a pair, 0.2 for three cards, 0.3 for three with one, 0.3 for three with two, 0.7 for three sequences, and 0.8 for a bomb, then the card scoring data can be calculated according to the score of each card type. The current card data is the current card, i.e., the card type with the highest score, for example, if the hand is 55556666, and the 55556666 card type has the highest score, then the current card data is 55556666. The self-game database is a database for storing the card data of each game in the self-game.

[0104] In a specific implementation, the current game information is converted into an encoding matrix to input into the reinforcement learning model to obtain all legal card types that can be played, the card types are scored according to the reward strategy, the card type with the highest score is found as the current card, and the card data is stored into the self-game database.

[0105] Step S304: updating the current game information, and determining whether the current game is over according to the updated current game information.

[0106] It should be understood that if the number of remaining cards of one player is 0, it means that the current game is over, and if the number of remaining cards of no player is 0, it means that the current game is still continuing and not over.

[0107] Furthermore, before the current hand game ends, based on the updated current hand game information, the step of inputting the current hand game information into the reinforcement learning model to obtain initial card-playing data is returned.

[0108] In practice, if the current game ends, the next step is taken; if the game is not over, the game continues until the current game ends.

[0109] Step S305: When the current game ends, obtain the game result data, determine the current sample data based on the game result data and the self-play database, and store the current sample data in the sample database.

[0110] It should be noted that the game result data refers to the current game result, such as: your side wins or your side loses. The current sample data refers to the sample data corresponding to the current game, with each card played and the result of the game forming one sample data entry.

[0111] In the actual implementation, after each game ends, the corresponding game result is generated, which, together with the data of each card played, forms a sample data and establishes a sample database.

[0112] Further, after step S30, the process includes: determining the number of samples based on the sample database; when the number of samples meets a preset cumulative number, training the reinforcement learning model based on the sample data in the sample database to obtain a new reinforcement learning model, and recording the number of iterations; when the number of iterations is greater than or equal to a preset iteration threshold, determining the game model based on the new reinforcement learning model.

[0113] It is understandable that when the number of samples does not meet the preset cumulative number, the self-play database is initialized and the process returns to step S302.

[0114] It should be understood that when the number of iterations is less than a preset iteration threshold, the self-play database is initialized, and the execution step S302 is returned according to the new reinforcement learning model.

[0115] In the specific implementation, if the number of samples in the sample database does not reach the sample number threshold, the self-play continues to generate sample data. If the number of model iterations is less than the preset iteration threshold, it is considered that the model has not yet reached the application standard, and the latest model is used to continue the self-play.

[0116] like Figure 6 The diagram shown illustrates the reinforcement learning process. Based on the game flow, game information, and card-playing rules, all possible legal card combinations are obtained. Card combinations are scored according to the reward design, and the card combination with the highest score is selected as the current card to play. When the data accumulates to a certain amount, the model is trained.

[0117] In this embodiment, a reinforcement learning model is established based on a preset neural network structure. The current game situation is initialized according to the game flow and rules to obtain the current game situation information. Based on the current game situation information, reward strategy, and reinforcement learning model, the current card-playing data is determined and stored in the self-play database. The current game situation information is updated. Based on the updated current game situation information, it is determined whether the current game situation has ended. When the current game situation ends, the game result data is obtained. Based on the game result data and the self-play database, the current sample data is determined and stored in the sample database. Using the current game situation information as input and the encoding matrix as the data format, a reinforcement learning model is designed based on CNN. The model predicts card play through the game environment, caches the game situation data generated by self-play, and combines each card play and the result of the game situation into a sample data. The accumulated sample data establishes a sample database for training the reinforcement learning model, helping to solve the "cold start" problem of the card game robot in new gameplay.

[0118] Furthermore, this embodiment of the invention also proposes a storage medium storing a card pattern prediction program based on model fusion, wherein when the card pattern prediction program based on model fusion is executed by a processor, it implements the steps of the card pattern prediction method based on model fusion as described above.

[0119] Reference Figure 7 , Figure 7 This is a structural block diagram of the first embodiment of the card pattern prediction device based on model fusion of the present invention.

[0120] like Figure 7 As shown, the card pattern prediction device based on model fusion proposed in this embodiment of the invention includes:

[0121] Environment building module 10 is used to build a game environment, which includes at least game rules and game flow.

[0122] The reinforcement learning module 20 is used to determine the reward strategy based on the game rules and preset requirements.

[0123] The reinforcement learning module 20 is also used to acquire current game information and establish a sample database based on the current game information, reward strategy and reinforcement learning model.

[0124] The reinforcement learning module 20 is also used to train the reinforcement learning model based on the sample database and establish a game model.

[0125] The reinforcement learning module 20 is also used to obtain game data based on the game model.

[0126] The supervision learning module 30 is configured to train a supervision learning model according to the game data, and obtain a card type prediction model.

[0127] The supervision learning module 30 is further configured to obtain prediction data according to the target game information and the card type prediction model, and make a move according to the prediction data.

[0128] In the embodiment, a game environment is constructed, a reward strategy is determined according to the game rules and preset requirements, current game information is obtained, a sample database is established according to the current game information, the reward strategy and the reinforcement learning model, the reinforcement learning model is trained according to the sample database, a game model is established, game data is obtained, a supervision learning model is trained according to the game data, a card type prediction model is obtained, prediction data is obtained according to the target game information and the card type prediction model, and a move is made according to the prediction data. The embodiment utilizes the characteristics of reinforcement learning from nothing to something, trains a robot model with high level first, generates massive game data based on the model, and then trains a card type classifier as the final robot model by using the game data generated by the reinforcement learning model as data samples based on the efficiency of supervision learning. Reinforcement learning and supervision learning are combined to effectively solve the “cold start” problem of the chess game robot in a new play, and greatly reduce the project operation cost.

[0129] In an embodiment, the reinforcement learning module 20 is further configured to establish the reinforcement learning model according to a preset neural network structure.

[0130] The current game is initialized according to the game flow and the game rules, and current game information is obtained.

[0131] The current move data is determined according to the current game information, the reward strategy and the reinforcement learning model, and the current move data is stored in the self-game database.

[0132] The current game information is updated, and whether the current game is over is determined according to the updated current game information.

[0133] When the current game is over, game result data is obtained, and current sample data is determined according to the game result data and the self-game database.

[0134] The current sample data is stored in the sample database.

[0135] In an embodiment, the reinforcement learning module 20 is further configured to convert the current game information into current game data according to a preset data format.

[0136] The current game data is input into the reinforcement learning model, and initial move data is obtained.

[0137] score the initial card data according to the reward policy, to obtain card score data;

[0138] determine current card data according to the card score data and the initial card data.

[0139] In an embodiment, the reinforcement learning module 20 is further configured to, when the current card game is not over, input the updated current card game information into the reinforcement learning model to obtain initial card data.

[0140] In an embodiment, the reinforcement learning module 20 is further configured to determine a sample quantity according to the sample database.

[0141] When the sample quantity meets a preset cumulative quantity, train the reinforcement learning model according to sample data in the sample database to obtain a new reinforcement learning model, and record an iteration number.

[0142] When the iteration number is greater than or equal to a preset iteration threshold, determine a game model according to the new reinforcement learning model.

[0143] In an embodiment, the reinforcement learning module 20 is further configured to, when the sample quantity does not meet the preset cumulative quantity, initialize the self-game database, and return to execute the step of initializing card game information according to the game flow and the game rules to obtain current card game information.

[0144] When the iteration number is less than the preset iteration threshold, initialize the self-game database, and return to execute the step of initializing card game information according to the game flow and the game rules to obtain current card game information according to the new reinforcement learning model.

[0145] In an embodiment, the supervised learning module 30 is further configured to establish a supervised learning model according to a preset neural network structure.

[0146] Preprocess the game data to obtain game matrix data and game label data.

[0147] Train the supervised learning model according to the game matrix data and the game label data to obtain a card type prediction model.

[0148] It should be understood that the above is only for illustration, and does not constitute any limitation on the technical solutions of the present application. In specific applications, those skilled in the art can set it up according to the needs, and the present application does not limit it.

[0149] It should be noted that the above-described workflow is merely illustrative and does not limit the scope of protection of the present application. In actual applications, a person skilled in the art can select part or all of the above-described workflow to achieve the purpose of the embodiment according to actual needs, which is not limited herein.

[0150] In addition, technical details not described in detail in the present embodiment can be found in the model fusion-based card type prediction method provided by any embodiment of the present application, which will not be described here.

[0151] In addition, it should be noted that in this document, the terms "comprise", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or system that includes the element.

[0152] The above-mentioned embodiment numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0153] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk) and includes a number of instructions to make a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0154] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A model fusion-based hand type prediction method, characterized in that, The model fusion-based card type prediction method comprises the following steps: A game environment is constructed, and the game environment at least comprises game rules and a game process; An award strategy is determined according to the game rules and preset requirements; Current card information is obtained, and a sample database is established according to the current card information, the award strategy and a reinforcement learning model, specifically comprising the following steps: a reinforcement learning model is established according to a preset neural network structure, a current card is initialized according to the game process and the game rules to obtain current card information, the current card information is converted according to a preset data format to obtain current card data, the current card data is input into the reinforcement learning model to obtain initial card data, the initial card data is scored according to the award strategy to obtain card score data, current card data is determined according to the card score data and the initial card data, and the current card data is stored in a self-play database, the current card information is updated, whether the current card is ended is determined according to the updated current card information, card result data is obtained when the current card is ended, current sample data is determined according to the card result data and the self-play database, and the current sample data is stored in the sample database; The reinforcement learning model is trained according to the sample database to establish a game model; Game data is obtained according to the game model and the game environment; The game data is used to train a supervised learning model to obtain a card type prediction model; Prediction data is obtained according to target card information and the card type prediction model, and the prediction data is used to make a card.

2. The method of claim 1, wherein, After the current card information is updated and whether the current card is ended is determined according to the updated current card information, the following steps are further included: When the current card is not ended, the updated current card information is used to return to the step of inputting the current card information into the reinforcement learning model to obtain initial card data.

3. The method of claim 1, wherein, The reinforcement learning model is trained according to the sample database to establish a game model, comprising the following steps: The number of samples is determined according to the sample database; When the number of samples meets a preset cumulative number, the reinforcement learning model is trained according to sample data in the sample database to obtain a new reinforcement learning model, and the number of iterations is recorded; When the number of iterations is greater than or equal to a preset iteration threshold, a game model is determined according to the new reinforcement learning model.

4. The method of claim 3, wherein, After the number of samples is determined according to the sample database, the following steps are further included: When the number of samples does not meet the preset cumulative number, the self-play database is initialized, and the step of initializing card information according to the game process and the game rules to obtain current card information is returned to be executed; After the number of samples meets the preset cumulative number, the reinforcement learning model is trained according to sample data in the sample database to obtain a new reinforcement learning model, and the number of iterations is recorded, the following steps are further included: When the number of iterations is less than a preset iteration threshold, the self-play database is initialized, and a step of initializing game information according to the game flow and the game rules and obtaining current game information according to the new reinforcement learning model is returned.

5. The method according to any one of claims 1 to 4, characterized in that, The step of training the supervised learning model according to the game data to obtain a card type prediction model comprises: establishing a supervised learning model according to a preset neural network structure; preprocessing the game data to obtain game matrix data and game label data; training the supervised learning model according to the game matrix data and the game label data to obtain a card type prediction model.

6. A model fusion-based hand prediction device, characterized by, The card type prediction device based on model fusion comprises: an environment construction module configured to construct a game environment, wherein the game environment at least comprises game rules and a game flow; a reinforcement learning module configured to determine a reward strategy according to the game rules and a preset requirement; the reinforcement learning module is further configured to obtain current game information, and establish a sample database according to the current game information, the reward strategy and a reinforcement learning model; the reinforcement learning module is further configured to train the reinforcement learning model according to the sample database to establish a game model; the reinforcement learning module is further configured to obtain game data according to the game model and the game environment; a supervised learning module configured to train a supervised learning model according to the game data to obtain a card type prediction model; the supervised learning module is further configured to obtain prediction data according to target game information and the card type prediction model, and make a card according to the prediction data; the reinforcement learning module is further configured to establish a reinforcement learning model according to a preset neural network structure, initialize a current game according to the game flow and the game rules to obtain current game information, convert the current game information according to a preset data format to obtain current game data, input the current game data into the reinforcement learning model to obtain initial card data, score the initial card data according to the reward strategy to obtain card score data, determine current card data according to the card score data and the initial card data, store the current card data into a self-play database, update the current game information, determine whether the current game is over according to the updated current game information, obtain game result data when the current game is over, determine current sample data according to the game result data and the self-play database, and store the current sample data into a sample database.

7. A model fusion-based hand type prediction device characterized by comprising: The device comprises a memory, a processor and a card type prediction program based on model fusion stored on the memory and executable on the processor, and the card type prediction program based on model fusion is configured to implement the steps of the card type prediction method based on model fusion in any one of claims 1 to 5.

8. A storage medium, characterized by The storage medium stores a card type prediction program based on model fusion, and the card type prediction program based on model fusion implements the steps of the card type prediction method based on model fusion in any one of claims 1 to 5 when executed by a processor.

Images