Interaction method, device and equipment based on card game robot and medium

Abstract

The application discloses an interaction method and device based on a card game robot, equipment and a medium, relates to the technical field of computers, and the game robot is a game virtual role controlled by an artificial intelligence algorithm. The method comprises the following steps: obtaining a target supervised learning model by analyzing a real game log PyTorch training supervised learning model; obtaining a target reinforcement learning model by training a reinforcement learning model comprising a card playing model and an action decision model in a simulation environment based on a distributed model-free control architecture and the log; in a target game, determining whether the current game belongs to a preset extreme game by a current game state information and a hand card prediction rule in the current game robot; and determining a current game strategy by a judgment result, the target supervised learning model in the current game robot, the target reinforcement learning model and a strategy fusion rule, so as to realize interaction with a player role in the game. The application can improve the humanization and rationality of the game robot.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to interactive methods, devices, equipment and media for card game robots. Background Technology

[0002] To enhance the gaming experience and meet the demand for matchmaking and gaming companionship, gaming platforms typically develop AI-powered gaming bots. Existing solutions for these bots mainly fall into three categories: (1) Implementation based on logical rules: The robot's card-playing is determined by writing a large number of rules and logic specific to the game. It is simple to implement, but the strategy is singular and easily recognized by real players, resulting in low levels of intelligence and anthropomorphism. (2) Implementation based on supervised learning: The robot is trained by collecting a large amount of game data from real players to imitate their decision-making patterns. Although the anthropomorphism is high, the intelligence and strategy level are limited by the quantity and quality of the training data, and no innovative playing styles will emerge. (3) Implementation based on reinforcement learning: The robot learns the optimal decision-making action by constructing a game simulation environment and conducting a large number of self-plays. However, real game data from real players is not used in the training, resulting in decisions that differ greatly from those of real players in certain states, i.e., the anthropomorphism is low. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide an interaction method, device, equipment, and medium for card-based gaming robots, which can improve the anthropomorphism, efficiency, rationality, and reliability of the gaming robot's gameplay while ensuring its intelligence and competitive level, thereby enhancing the gaming experience for real players. The specific solution is as follows: Firstly, this application provides an interaction method based on a card-based gaming robot, wherein the gaming robot is a virtual game character controlled by an artificial intelligence algorithm, used to interact with the player character during a game. The method includes: The real game logs of the online game server are analyzed, and the obtained samples and player whitelist are used to train the supervised learning model in the game bot using PyTorch to determine the trained target supervised learning model; wherein, the real game logs are the game logs of real player characters. The reinforcement learning model in the game robot is trained in a simulated environment using a distributed model-free control architecture and the real game logs to determine the trained target reinforcement learning model; the reinforcement learning model includes a card-playing model and an action decision model. In the target game, the hand prediction rules in the current game robot are used to determine whether the current target game belongs to a preset extreme game, and the current game status information is used to determine the judgment result; Based on the judgment result, the target supervised learning model, the target reinforcement learning model, the hand prediction rules, and the strategy fusion rules in the current game robot, the current game strategy is determined so as to realize game interaction according to the current game strategy.

[0004] Optionally, the parsing of the actual game logs from the online game server includes: Use the game log table to read the actual game logs from the online game server; The actual game logs are processed based on the deduplication flow to determine the processed game logs; The processed game log is parsed in parallel based on the target state machine to determine the log parsing result; The processed game logs are classified according to the log parsing results to determine the log classification results; the log classification results include several model samples and several player win rate samples; the model samples include card-playing decisions and action decisions; Based on the player win rate sample, analyze the player character's game level to determine the level analysis results; The player whitelist is determined based on the results of the horizontal analysis.

[0005] Optionally, the step of using the obtained samples and player whitelist to train the supervised learning model in the game bot using PyTorch to determine the trained target supervised learning model includes: Construct the player whitelist into a Python set all at once; The model samples are sampled based on the Python set to determine the sampling results; The training and validation datasets are determined based on the sampling results; The proportion of decision categories in the samples of the training dataset is analyzed to obtain the category proportion analysis results; The training dataset is downsampled based on the category proportion analysis results to determine the processing results; The processing results are shuffled at the process level using a multi-process data loader to determine the target training set. The supervised learning model in the game robot is trained using PyTorch based on the target training set until the training termination condition is met using the validation dataset, thus obtaining the trained target supervised learning model. The target supervised learning model is exported based on a preset protocol to obtain a model file.

[0006] Optionally, the step of training the reinforcement learning model in the game robot in a simulated game environment based on the distributed model-free control architecture and the real game logs to determine the trained target reinforcement learning model includes: Obtain the initial reinforcement learning model in the game robot; A shared memory buffer is created based on a distributed model-free control architecture; The participant process pool and learner process pool are initialized based on the distributed model-free control architecture to obtain participant processes and learner processes. In the early stage of training, based on the first information extraction index, the last few card-playing information of each player character before winning the hand in the real game logs is extracted to obtain the initial training dataset. The reinforcement learning model is launched based on the initial training dataset to conduct self-game training in a simulated environment in order to obtain the game results of the current self-game training round. The self-play training data is collected based on the participant process, and the collected self-play training data is saved to the shared memory buffer; the self-play training data includes the game results; The learner process reads the self-play training data from the shared memory buffer to determine the reading result; The learner process and the reading results are used to model a Markov decision process to determine the modeling results. The policy gradient and model parameters are updated using the learner process and the modeling results to determine the update results; Based on the updated results, the reinforcement learning model continues to be trained in a simulated game environment until the mid-training stage, at which point the model performance data of the current reinforcement learning model is obtained. The second information extraction index is determined based on the model performance data; Based on the second information extraction index, the last few card-playing information of each player character before winning the hand in the real game log is extracted from the real game log to obtain the updated target training dataset. The current reinforcement learning model is trained in a simulated game environment based on the target training dataset, the participant process, and the learner process until a preset loop termination condition is met, at which point the trained target reinforcement learning model is determined.

[0007] Optionally, the step of determining whether the current target game match belongs to a preset extreme match by using the hand prediction rules in the current game robot and the current game state information includes: The current game bot obtains the current game status information of the target game from the online game server. The game bot records the current historical behavior information of each player character in the target game; wherein, the current historical behavior information includes historical card playing information; The remaining card information is obtained by recording the information of the remaining cards outside the target game session using the current game robot. Using the current game bot, and utilizing current historical behavior information, current remaining card information, current game state information, and hand prediction rules, the hand prediction results for each player character in the target game are determined; The probability of winning is determined by the current game robot and the hand prediction results. The first judgment result is obtained by determining whether the current target game match belongs to a preset extreme match based on the current game robot, the winning probability, the preset upper limit threshold, and the preset lower limit threshold.

[0008] Optional, also includes: The second judgment result is obtained by determining whether the current target game game belongs to the preset extreme game by using the current game robot, current historical behavior information, current remaining card information and current game status information.

[0009] Optionally, determining the current game strategy based on the judgment result, the target supervised learning model in the current game robot, the target reinforcement learning model, the hand prediction rule, and the strategy fusion rule includes: The basic state vector is obtained by performing single-hot vector encoding on the current game robot, the hand prediction result, player role information, dealer information, current historical behavior information, current remaining card information, game stage information, and consecutive dealer information; A first strategy is determined using the target reinforcement learning model and the basic state vector in the current game robot; the first strategy includes initial card-playing decision probability information and initial action decision probability information. The second strategy is determined by using the target-supervised learning model in the current game bot and by utilizing the current game state information; The current strategy fusion parameters are determined by the current game robot, the first judgment result, and the second judgment result. The current game strategy is determined by the current game robot, the current strategy fusion parameters, the first strategy, and the second strategy.

[0010] Secondly, this application provides an interactive device based on a card-based gaming robot, wherein the gaming robot is a virtual game character controlled by an artificial intelligence algorithm, used to interact with the player character during a game. The device includes: The first model training module is used to parse the real game logs of the online game server, and use the obtained samples and player whitelist to train the supervised learning model in the game robot using PyTorch, so as to determine the trained target supervised learning model; wherein, the real game logs are the game logs of real player characters. The second model training module is used to train the reinforcement learning model in the game robot in a simulated environment based on the distributed model-free control architecture and the real game logs, so as to determine the trained target reinforcement learning model; the reinforcement learning model includes a card playing model and an action decision model. The game judgment module is used to determine whether the current target game game belongs to a preset extreme game game by using the hand prediction rules in the current game robot and the current game state information in the target game game, so as to determine the judgment result; The strategy fusion module is used to determine the current game strategy based on the judgment result, the target supervised learning model in the current game robot, the target reinforcement learning model, the hand prediction rule, and the strategy fusion rule, so as to realize game interaction according to the current game strategy.

[0011] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the aforementioned interactive method for a card-based game robot.

[0012] Fourthly, this application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the steps of the aforementioned interactive method for a card-based game robot.

[0013] As can be seen, in this application, the real game logs of the online game server are analyzed, and the obtained samples and player whitelist are used to train the supervised learning model in the game robot using PyTorch to determine the trained target supervised learning model; wherein, the real game logs are game logs of real player characters; based on the distributed modelless control architecture and the real game logs, the reinforcement learning model in the game robot is trained in a simulated environment using self-game training to determine the trained target reinforcement learning model; the reinforcement learning model includes a card playing model and an action decision model; in the target game game, the hand prediction rules in the current game robot are used, and the current game state information is used to determine whether the current target game game belongs to a preset extreme game to determine the judgment result; based on the judgment result, the target supervised learning model in the current game robot, the target reinforcement learning model, the hand prediction rules and the strategy fusion rules, the current game strategy is determined, so as to realize game interaction according to the current game strategy. In other words, this application first obtains samples and a player whitelist by parsing real game logs, then trains the supervised learning model in the game bot using PyTorch to obtain the target supervised learning model. Next, based on a distributed model-free control architecture and real game logs, the reinforcement learning model of the game bot, which includes a card-playing model and an action decision model, is trained in a simulated game environment to obtain the target reinforcement learning model. Then, in the target game, the current game state information and the current game bot's hand prediction rules are used to determine whether it belongs to a preset extreme game. Finally, based on the judgment result, the strategies output by the target supervised learning model and the target reinforcement learning model in the current game bot are merged, and the obtained current game strategy is used to realize the game interaction in the target game. In this way, while ensuring the intelligence and competitive level of the game bot's game, the anthropomorphism, efficiency, rationality, and reliability of the game bot's game can be improved, thereby enhancing the gaming experience for real players. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] Figure 1 A flowchart illustrating an interaction method for a card-based game robot provided in this application; Figure 2 This application provides a specific decision-making flowchart for a card game robot. Figure 3A flowchart illustrating the training process of a supervised learning model provided in this application; Figure 4 A schematic diagram of the structure of a reinforcement learning model provided in this application; Figure 5 A schematic diagram of the training process of a reinforcement learning model provided in this application; Figure 6 A schematic diagram of an interactive device structure for a card game robot provided in this application; Figure 7 This application provides a structural diagram of an electronic device. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] In the existing related solutions, the implementation of game robots is mainly divided into three categories: (1) Implementation based on logic rules: The robot decides to play cards by writing a large number of rules and logic for this game. It is simple to implement, but the strategy is simple and it is extremely easy to be identified by real players. The degree of intelligence and anthropomorphism is low. (2) Implementation based on supervised learning: The robot is trained by collecting a large amount of game data from real players and imitating the decision-making mode of real players. Although the degree of anthropomorphism is high, the intelligence and strategy level are limited by the quantity and quality of training data and no innovative playing methods will appear. (3) Implementation based on reinforcement learning: The robot learns the optimal decision action by constructing a game simulation environment and conducting a large number of self-games. However, the real game data of real players is not used in the training, which leads to the decision action in some states being very different from the decision action of real players, that is, the degree of anthropomorphism is low.

[0018] Therefore, this application provides an interaction scheme for a card game robot, which can improve the anthropomorphism, efficiency, rationality and reliability of the game robot while ensuring the intelligence and competitive level of the game robot, thereby enhancing the gaming experience of real players.

[0019] See Figure 1 As shown in the figure, this invention discloses an interaction method based on a card-based gaming robot. The gaming robot is a virtual game character controlled by an artificial intelligence algorithm, used to interact with the player character during a game. The method includes: Step S11: Analyze the real game logs of the online game server, and use the obtained samples and player whitelist to train the supervised learning model in the game robot using PyTorch, so as to determine the trained target supervised learning model; wherein, the real game logs are the game logs of real player characters.

[0020] In this embodiment, combined with Figure 2 As shown, the game robot includes a supervised learning model and a reinforcement learning model, namely a hand prediction model, which ultimately outputs the optimal action to the game server.

[0021] Specifically, to train the supervised learning model in the game robot, real game logs are first parsed to obtain samples and a whitelist. This involves: reading real game logs from the online game server using a game log table; processing the real game logs based on deduplication flow to determine the processed game logs; performing parallel parsing of the processed game logs based on a target state machine to determine the log parsing result; classifying the processed game logs according to the log parsing result to determine the log classification result; the log classification result includes several model samples and several player win rate samples; the model samples include card-playing decisions and action decisions; analyzing the player character's game level based on the player win rate samples to determine the level analysis result; and determining the player whitelist based on the level analysis result.

[0022] It is important to understand that regarding log parsing and sample construction, the original data is the game log from the Mahjong server ( The log file (.log) uses chardet (a Python third-party library for automatically detecting text encoding) to automatically detect the log encoding and reads it according to the gb18030 encoding standard. After deduplication (data stream), the game state is replayed line by line. Log parsing is completed by a self-developed state machine LogGameTb. When the game reaches the SuccessfulEnd state, the main model samples (i.e., model samples, including ordinary card playing information and action decision information) and player win rate samples are separated into three types of buffers. Each sample consists of the observation tensor obs and the action label label. The samples are persisted in files in JSON Lines format (Javascript ObjectNotation Lines, a text data format) for easy random splitting later. This stage uses multiprocessing.Pool (the core class in the Python standard multiprocessing used to implement the process pool Pool) to parse the log in parallel (number of processes n-proc=64) to significantly reduce the log parsing time.

[0023] Regarding player selection, this embodiment considers that mahjong strategies heavily rely on player skill levels, and uses qualified players as the selection unit. The win rate of each player is aggregated. r Number of games N With data volume D Only retain those that satisfy: The player whitelist (uid_list.xlsx, approximately a thousand players) is generated by selecting players (the threshold was determined experimentally, taking into account both data quality and scale).

[0024] Furthermore, in this embodiment, the aforementioned offline log observations are reconstructed into a game_tb structure that is isomorphic to the online deployment, and then further serialized into a fixed shape. The tensor, in which T =32 (sequence length) F =36 (channel dimension), and the two are strictly consistent in the training and inference stages, ensuring zero offset between offline training and online service representation.

[0025] In this embodiment, after obtaining the samples and the player whitelist, the supervised learning model in the game bot will be trained using PyTorch (a deep learning framework). To handle long-tail distributions during training, a streaming dataset MJDataset is implemented based on torch.utils.data.IterableDataset (an abstract base class in PyTorch for handling iterable datasets). Specifically: the player whitelist is constructed as a Python set; the model samples are sampled based on the Python set to determine the sampling results; the training dataset and validation dataset are determined based on the sampling results; the decision category proportions of the samples in the training dataset are analyzed to obtain category proportion analysis results; the training dataset is downsampled based on the category proportion analysis results to determine the processing results; the processing results are shuffled at the process level using a multi-process data loader to determine the target training set; the supervised learning model in the game bot is trained using PyTorch based on the target training set until the training termination condition is met using the validation dataset, resulting in a trained target supervised learning model; the target supervised learning model is exported based on a preset protocol to obtain the model file. Understandably, the training and validation sets are split at a file-level ratio of 9:1, and the validation set is not downsampled or shuffled to ensure that the evaluation metrics are comparable.

[0026] It is important to understand that, in combination Figure 3As shown, regarding the PyTorch training of the supervised learning model, during sample collection: 1) UID lookup is accelerated: the whitelist is constructed as a Python set all at once, and the complexity of filtering for each sample's UID (UserIdentifier) ​​is reduced from... Down to 2) Category-related downsampling: Samples of the over / inactive class, which account for about 22%, are retained at P=0.25. Samples of the response action group (accounting for 35≤C≤41) are retained in full. The remaining classes are retained at P=0.75. 3) Worker-level shuffle: Under the multi-process DataLoader, each worker (process) maintains a reserveoir shuffle buffer with a capacity of 4096 to break the correlation of samples in the same batch and stabilize the gradient direction. 4) Robust reading: JSON parsing errors and single sample parsing anomalies in the above process are downgraded and skipped to avoid bad data causing training interruption.

[0027] Furthermore, regarding the network structure of the supervised learning model, the main model is based on a Transformer Encoder, implemented using PyTorch's nn.TransformerEncoder (a class provided in PyTorch's torch.nn module for implementing stacked transformer encoders). Given input... The forward computation is as follows: ; ; ; ; in, This represents the original feature vector of the input layer; This represents the hidden state of the Lth layer network at the initial time step; This represents the converter encoder of layer L; y The output is represented by B, where B is the batch size, T is the sequence length (time steps), and F is the input feature dimension. The weight matrix represents the input linear transformation, and has a size of F×d; The bias vector representing the input linear transformation has a shape of B×T×d; This represents the feature vector after adding positional encoding, with a shape of B×T×d; The positional encoding depends on the sequence length T and is typically in the shape of T×d. This represents the weight matrix of the output classification layer, with a size of C×d, which maps the d-dimensional features output by the encoder to C categories; This represents the bias vector of the output classification layer, with dimension C; Let represent the C-dimensional real vector space, and let represent the space containing the class probability vectors generated by the output layer.

[0028] Furthermore, the linear embedding layer stem: nn.Linear(36, 512), increases the dimension of each time step channel from 36 to 512. Sine positional encoding: Referencing the non-trainable sin / cos encoding of Vaswani et al. (2017), word embedding dimension d_model=512, maximum sequence length max_len=100, with added dropout regularization (p=0.1). Transformer encoder: L=4 encoder layers, each with word embedding dimension d_model=512, multi-head attention head number h=8, feedforward hidden layer dimension d_ff=768, built-in dropout regularization=0.2. CLS-style classification head: represented by the first token. The logits of C=44 classes are obtained by nn.Linear(512, C). The model has approximately 1.26 × 10^6 parameters. 7 Before input, the tensor undergoes an axis transformation (transpose(1,0)) to adapt to the early semantic conventions of PyTorch (seq_len, batch, feature). Here, (seq_len, batch, feature) represents the dimensionality format of the input tensor, i.e., (sequence length, batch size, feature dimension).

[0029] Regarding the training strategy for supervised learning models, 1) the loss function uses Label Smoothing Cross-Entropy (a self-developed label smoothing cross-entropy) to alleviate label noise and suppress overfitting, as shown in the following formula: .

[0030] In the formula, This represents the soft target probability corresponding to the c-th category after label smoothing. It is no longer in one-hot form, but rather a partial probability quality is evenly distributed across all categories to alleviate label noise and suppress overfitting. This represents the total number of categories in the classification task (C=44). This represents the smoothing factor.

[0031] 2) The optimizer uses torch.optim.AdamW (an improved version of the Adam optimizer with weight decay), and the learning rate... Weight decay The learning rate scheduling uses ReduceLROnPlateau (a popular learning rate scheduler) (parameters: mode='max', learning rate reduction factor=0.1, patience=3, lower bound of learning rate min_lr=). The validation set accuracy is used as the monitoring metric. Here, `patience=n` indicates that the maximum number of epochs before triggering a decrease in the learning rate is n epochs; if there is no improvement after n consecutive epochs, a decrease in the learning rate will be triggered. `mode='max'` means that the higher the monitoring metric (e.g., accuracy), the better. The scheduler will wait until the metric stops increasing before decreasing the learning rate, rather than stopping training. The actual control over training termination is early stopping.

[0032] 3) Training hyperparameters: batch size = 4096, number of threads = 8, maximum number of training epochs = 200. Hardware can use a single NVIDIA GPU (CUDA: 0), with the boolean parameter pin_memory = True to reduce the copy overhead from the CPU to the GPU. In the validation phase, in addition to the standard multi-class accuracy torchmetrics.Accuracy(task='multiclass'), it also implements compute_per_class_accuracy for each class, with results synchronously written to TensorBoard (a visualization toolkit) and Excel for easy tracking of training dynamics for long-tail classes.

[0033] To facilitate the export and inference deployment of the trained target-supervised learning model and meet the low-dependency inference requirements of the online C++ / Go backend, ONNX + ONNX Runtime (Open Neural Network Exchange) is adopted as the cross-language deployment protocol. The export process is as follows: 1) Load the optimal checkpoint using torch.load and call model.eval() to fix training-phase operators such as BN / Dropout; 2) Construct a pseudo-input of shape (1,T,F)=(1,32,36), and export it through the PyTorch model transformation entry function torch.onnx.export. Key parameters: opset_version=17 (overrides the latest Transformer-related operators); do_constant_folding=True (speeds up constant folding); dynamic_axes={'input':{0:'batch_size'}, 'output':{0:'batch_size'}} (Dynamic batch size, supports arbitrary batch size online); 3) Use onnx.checker.check_model (used to check whether the model conforms to the ONNX standard specification) to verify the validity of the graph structure; 4) Numerical Consistency Verification: 100 random inputs are constructed and inferred using the original PyTorch supervised learning model and the core interface of ONNXRuntime used to load and run ONNX format machine learning models: onnxruntime.InferenceSession(CPUExecutionProvider). For each output, torch.testing.assert_close is called to determine tensor numerical equivalence, confirming that the cross-framework inference results are consistent within the floating-point error range, thus eliminating derived ambiguities. The final output is the ONNX format model file.

[0034] Step S12: Based on the distributed model-free control architecture and the real game logs, the reinforcement learning model in the game robot is trained in a simulated game environment to determine the trained target reinforcement learning model; the reinforcement learning model includes a card playing model and an action decision model.

[0035] In this embodiment, the DMC framework is used to train the reinforcement learning model in the game robot. The DMC framework stands for Distributed Model-free Control, a distributed model-free control reinforcement learning framework. Its core advantage is that it achieves parallel training on multiple devices through the Actor-Learner separation architecture, thereby improving training efficiency and adapting to large-scale self-play data collection and model updates. That is: Obtain the initial reinforcement learning model in the game robot; create a shared memory buffer based on a distributed model-free control architecture; initialize the participant process pool and learner process pool based on the distributed model-free control architecture to obtain participant processes and learner processes; in the early stage of training, based on the first information extraction metric, extract the last few card-playing information of each player character before winning from the real game logs to obtain the initial training dataset; start the reinforcement learning model to perform self-game training in a simulated environment based on the initial training dataset to obtain the game results of the current self-game training round; collect self-game training data based on the participant processes and save the collected self-game training data to the shared memory buffer; the self-game training data includes the game results; use the learner process to read the self-game training data from the shared memory buffer to... The process involves: determining the reading result; using the learner process and the reading result to perform Markov decision process modeling to determine the modeling result; using the learner process and the modeling result to update the policy gradient and model parameters to determine the update result; continuing to train the reinforcement learning model in a simulated game environment based on the update result until the mid-training stage is reached, obtaining the model performance data of the current reinforcement learning model; determining a second information extraction index based on the model performance data; extracting the last few card-playing information of each player character before winning from the real game logs based on the second information extraction index, to obtain the updated target training dataset; and training the current reinforcement learning model in a simulated game environment based on the target training dataset, the participant process, and the learner process until a preset loop termination condition is met, thus determining the trained target reinforcement learning model.

[0036] It is important to understand that, in combination Figure 4As shown, the reinforcement learning model differs from traditional single-model approaches by employing a binary classification decision architecture with two models (a card-playing model and an action decision model). This decouples the fundamentally different decision logics of "whether to perform special actions such as eating, ponging, konging, or winning" from "which card to play when performing a card-playing action," significantly reducing the action space complexity that a single model would need to learn. Furthermore, the card-playing model includes a value branch for analyzing state value and a decision branch for analyzing execution probability. Its input layer is 1713-dimensional, comprising 1577 dimensions of basic states and 34 candidate card-playing actions. The action decision model is 4-dimensional. It consists of an input layer, three hidden layers (each containing 512 neurons), and a value-head output layer that outputs the state value. The input layer is 1583-dimensional, including 1577 dimensions of basic states and 6 dimensions of action decision information. The action decision information and candidate card-playing actions are empty at input.

[0037] Furthermore, regarding the training of reinforcement learning models, combined with Figure 5 As shown, a distributed training architecture is adopted: Actor processes (participant processes) and Learner threads (learner processes), using a shared memory buffer for data exchange. After each self-play game, a definite game result (winning, draw, etc.) is generated, and the model is rewarded based on this final result.

[0038] During training, the Mahjong decision process can be represented as a quintuple using Markov Decision Process (MDP) modeling: Where: S is the state space (i.e., the encoded 1577-dimensional state vector); A is the action space (including playing cards, eating, ponging, konging, winning, and passing); P is the state transition probability; R γ is the immediate reward function; γ is the discount factor, γ∈[0,1].

[0039] Meanwhile, regarding the definitions of the value function and the Q function, the action-value function... This represents the expected cumulative reward that can be obtained by performing action a in state s and following policy π: .

[0040] in, E Expressing expectations; The immediate reward obtained at time step t+k+1; k This is the time step offset, starting from 0, representing the k-th future step; t This is the current time step; is the discount factor raised to the power of k, used to calculate the discount factor for the reward in the k-th step of the future.

[0041] The optimal action-value function satisfies the Bellman optimality equation, as shown below: .

[0042] in, Let represent the optimal action value function, which represents the expected cumulative discount reward that can be obtained by following the optimal strategy after performing action a in state s. Indicates the next state Possible actions to take at that time; Indicates the next state All possible actions below Take the maximum value, which is the optimal action; Indicates the next state Execute action The optimal action value afterward; This represents the immediate reward function, which indicates the immediate reward obtained after performing action a in state s; Represents the execution of actions The next state after that.

[0043] In addition, regarding policy gradients and model updates, model parameters... Update by minimizing the temporal difference (TD) loss: .

[0044] in, For parameters The loss; Rewards are given for each step taken immediately. For parameters The target value y is the action value prediction output by the neural network (Q-network). .

[0045] The target network parameters are periodically copied from the online network.

[0046] Furthermore, in the distributed training architecture, the number of Actor processes matches the number of image processor devices (num_actor_devices=4 in the training parameters, corresponding to 4 Actor processes), the number of Learner threads is 4 (i.e., num_threads=4), and the number of shared memory buffers is 200 (i.e., num_buffers=200), used to store the self-play data collected by the Actor processes. The Learner threads read data from the buffers to update the model. The reward calculation rules are as follows: winning a hand earns a positive reward (the specific value can be adjusted according to the difficulty of the game, such as a basic reward of 10 points), a draw earns a neutral reward (0 points), losing a hand or discarding a winning tile earns a negative reward (-n points), and consecutive wins add an extra reward (2 points for each consecutive win). This reward mechanism ensures that the model can learn the optimal winning strategy.

[0047] Furthermore, in this embodiment, when training the reinforcement learning model, a key segment sampling-based training method is adopted. In the initial training phase, instead of using all game data, the last 5 moves before each player wins are selectively extracted from a large number of winning games as the initial training dataset. During and after the training phase, the training data is changed to the last 8 moves before winning or all game data, depending on the model's learning performance. This improves training efficiency and convergence speed. All input features are vectorized using one-hot encoding, ensuring that the neural network can accurately identify various discrete state information and avoid information loss in feature representation.

[0048] Step S13: In the target game, the hand prediction rules in the current game robot are used to determine whether the current target game belongs to the preset extreme game by using the current game state information, so as to determine the judgment result.

[0049] In this embodiment, when the trained game robot is applied to a game, the hand prediction rule in the robot (i.e., Figure 2The hand prediction model in the game infers the hand distribution of opponents in real time based on their historical behavior (playing cards, eating, ponging, and konging) to determine how to integrate the strategies output by the target reinforcement learning model and the target supervised learning model. Specifically: the game bot obtains the current game state information of the target game from the online game server; the game bot records the current historical behavior information of each player character in the target game, including historical card playing information; the game bot records the information of the remaining cards outside the game to obtain the current remaining card information; the game bot, using the current historical behavior information, the current remaining card information, the current game state information, and the hand prediction rules, determines the hand prediction result for each player character in the target game; the game bot and the hand prediction result determine the probability of winning; the game bot, the probability of winning, a preset upper threshold, and a preset lower threshold determine whether the current target game belongs to a preset extreme game, obtaining a first judgment result. Understandably, the core function of a hand prediction model is card counting and prediction. By recording all players' played cards, melds, and kongs, and combining this with information on the remaining cards, it can predict the possible combinations of each player's hand (such as predicting the possible winning hand of a player).

[0050] Understandably, the preset extreme game scenarios include sure-win scenarios and loser scenarios. The criteria for judgment are as follows: the robot calculates the current probability of winning by using the hand prediction model and the information of the remaining cards. If the probability of winning is ≥90%, it is considered a sure-win scenario. If the probability of winning is ≤10%, it is considered a loser scenario. Conversely, if 90% > probability of winning > 10%, it is considered a normal game scenario.

[0051] Furthermore, under specific circumstances, the judgment of the preset extreme game situation also includes scenarios such as: a player has made ≥3 consecutive melds (chow, pong, kong), the number of remaining tiles is ≤5, and the current player is ready to win but has no chance of winning with the remaining tiles. After triggering fusion, the decision weights of the supervised learning model are added to the policy result provided by the reinforcement learning model. That is: by judging whether the current target game situation belongs to the preset extreme game situation based on the current game robot, current historical behavior information, current remaining tile information, and current game state information, a second judgment result is obtained.

[0052] Step S14: Based on the judgment result, the target supervised learning model, the target reinforcement learning model, the hand prediction rule, and the strategy fusion rule in the current game robot, determine the current game strategy, so as to realize game interaction according to the current game strategy.

[0053] In this embodiment, after determining the judgment result, the strategies output by the target reinforcement learning model and the target supervised learning model in the robot are fused based on the result, and the final game strategy is output to the game server. That is: a basic state vector is obtained by performing one-hot vector encoding on the current game robot, the hand prediction result, player role information, dealer information, current historical behavior information, current remaining card information, game stage information, and consecutive dealer information; a first strategy is determined by the target reinforcement learning model in the current game robot and the basic state vector; the first strategy includes initial card playing decision probability information and initial action decision probability information; a second strategy is determined by the target supervised learning model in the current game robot and using the current game state information; current strategy fusion parameters are determined by the current game robot, the first judgment result, and the second judgment result; and the current game strategy is determined by the current game robot, the current strategy fusion parameters, the first strategy, and the second strategy. The card playing model output consists of two tensors: a value head and an auxiliary head. The tensor output by the value head is [batch, 1], representing the current state and the value of the candidate cards; the tensor output by the auxiliary head is [batch, 4], used for multi-task assistance.

[0054] Understandably, the strategy fusion mechanism is used in late-game situations or in pre-set extreme game scenarios (win-or-lose situations). Based on the labels ultimately predicted by the supervised learning model, it fuses the optimal values ​​into the decision results of the reinforcement learning model, generating more human-like decision outcomes. In regular games, the highly skilled reinforcement learning model primarily dominates, ensuring the AI's high-order strategies and win rate. However, in late-game situations where reinforcement learning struggles to make effective decisions, or in win-or-lose situations, the system can accurately identify and introduce a highly human-like supervised learning model for decision intervention. The strategy fusion mechanism introduces fused decision values... for: .

[0055] in, The strategy output by the reinforcement learning model for the target; 'a' represents the policy output by the supervised learning model for the target; 'a' represents the dynamic fusion parameter. At that time, the regular bureau focused on strengthening learning; In extreme situations, supervision and assistance are the primary functions.

[0056] It is important to understand that, regarding the encoding of the basic state vector, taking Mahjong as an example, the encoding scheme used in this embodiment is as follows: 1) Hand information encoding: using a 34 It is represented by 4-dimensional one-hot vector encoding. There are a total of 9 different hand cards. There are 34 types of cards, with a maximum of 4 of each type. Each dimension of this vector corresponds to the number of a particular type of card; for example, 0 cards is represented by 0000, and 1 card is represented by 1000.

[0057] 2) Current player information encoding: A 5-dimensional vector is used to represent the current player's relative position, specifically the current player, the next player, the opposite player, the previous player, and none.

[0058] 3) Dealer Information Encoding: A 5-dimensional vector is used to represent the dealer's relative position, which makes it easier for the current player to judge the position relationship between the dealer and themselves. The specific position is consistent with the current player's information encoding, which is: current player, next player, opposite player, previous player, none.

[0059] 4) Encoding of each player's "eat," "pong," and "kong" information: using four sets of 34. A 4-dimensional one-hot vector represents the information about each player's current actions (eat, pong, kong). It records the information from the start of the game to the current state, sequentially recording all the eat, pong, kong actions for each player in relative order (1 for each player). 34 4).

[0060] 5) Card information encoding for each player: using 4 34 A 4-dimensional one-hot vector represents the card information played by each player. It records the card information played by each player from the start of the game to the current state, sequentially recording the cards played by each player according to their relative positions (1 for each player). 34 4).

[0061] 6) Encoding of each player's hand size: using 4 A 14-dimensional vector represents the number of cards in each player's hand. The state of each player's current hand count is recorded sequentially according to their relative positions.

[0062] 7) Encoding of the previous player's information: A 5-dimensional vector is used to represent the seat number of the player who played the last card, with the specific positions being the current player, the next player, the opposite player, the previous player, and none.

[0063] 8) Toad information encoding: using 1 34 A 4-dimensional one-hot vector represents the wild card information in this game, which helps the model to understand and learn the meaning of wild cards more clearly.

[0064] 9) Remaining card information encoding: using 1 34 A 4-dimensional one-hot vector records information about the remaining cards in the current round, essentially acting as a card counter. The model can use this feature to understand which cards remain in the game, facilitating subsequent decision-making.

[0065] 10) Game phase information encoding: Use an 8-dimensional vector to record the current state, which is divided into: drawing a card, playing a card, eating a card (left eating, middle eating, right eating), ponging a card, konging a card, and winning a hand.

[0066] 11) Encoding of consecutive wins: Use a 2D vector to record the current consecutive wins of the dealer in this round. Scalar normalization, if the consecutive wins exceed two rounds, it is counted as two rounds.

[0067] All of the above features are represented using one-hot encoding. The vectors of all the above parts are concatenated together to form a complete, high-dimensional basic state vector, which serves as the final input to the neural network.

[0068] In summary, this embodiment constructs a card game robot for online card games and intelligent card game machines, which combines high competitiveness, high anthropomorphism, high decision-making efficiency, strong logic, and efficient training characteristics, thereby improving the player experience during the game.

[0069] Therefore, in this embodiment, samples and a player whitelist are first obtained by parsing real game logs to train the supervised learning model in the game bot using PyTorch, resulting in a target supervised learning model. Then, based on a distributed model-free control architecture and real game logs, the reinforcement learning model of the game bot, which includes a card-playing model and an action decision model, is trained in a simulated game environment to obtain a target reinforcement learning model. Next, in the target game, the current game state information and the current game bot's hand prediction rules are used to determine whether it belongs to a preset extreme game. Finally, based on the judgment result, the strategies output by the target supervised learning model and the target reinforcement learning model in the current game bot are merged, and the obtained current game strategy is used to realize game interaction in the target game. In this way, while ensuring the intelligence and competitive level of the game bot's game, the anthropomorphism, efficiency, rationality, and reliability of the game bot's game are improved, thereby enhancing the gaming experience for real players.

[0070] As a preferred embodiment, the strategy fusion under preset extreme game conditions can be replaced with a dynamic weight fusion mechanism: by setting adaptively adjustable weight coefficients, the decision weights of the reinforcement learning model and the supervised learning model are dynamically allocated according to the game stage (early, mid, and late). For example, in the early game, reinforcement learning weights are dominant (accounting for 70%-80%) to ensure the high-order nature of the competitive strategy; in the mid-game, the weights are balanced (each accounting for 50%) to take into account both competitiveness and human-likeness; in the late game, supervised learning weights are dominant (accounting for 70%-80%) to ensure that the card-playing logic conforms to human habits. This alternative solution is not limited to the preconditions of preset extreme game conditions and can achieve adaptive fusion throughout the entire game stage, achieving the same effect of balancing high competitiveness and high human-likeness, and is adaptable to more game scenarios.

[0071] As a preferred embodiment, the binary classification decision architecture of the dual-model (card-playing model and action decision model) reinforcement learning model in the aforementioned embodiments can be replaced with an architecture including a single model and an action classifier. That is, a single reinforcement learning model is retained, but an action classifier is added to the model output. The decision action output by the model is first distinguished by the classifier between actions such as eating, ponging, konging, and winning, and actions like playing cards. Then, the corresponding decision logic is called for different types of actions (which can be a separately trained action judgment submodule). In this way, there is no need to split the reinforcement learning model; the action type can be separated through the classifier. This also reduces the decision dimension and computational complexity, allowing the model to accurately learn the triggering conditions and value weights of different actions, achieving the same technical effect as the dual-model architecture, and simplifying the model deployment process.

[0072] See Figure 6 As shown in the illustration, this application also discloses an interactive device based on a card game robot. The game robot is a virtual game character controlled by an artificial intelligence algorithm, used to interact with the player character during a game. The device includes: The first model training module 11 is used to parse the real game logs of the online game server, and use the obtained samples and player whitelist to train the supervised learning model in the game robot using PyTorch, so as to determine the trained target supervised learning model; wherein, the real game logs are the game logs of real player characters. The second model training module 12 is used to train the reinforcement learning model in the game robot in a simulated environment based on the distributed model-free control architecture and the real game logs, so as to determine the trained target reinforcement learning model; the reinforcement learning model includes a card playing model and an action decision model. The game judgment module 13 is used to determine whether the current target game game belongs to a preset extreme game game by using the hand prediction rules in the current game robot and the current game state information in the target game game, so as to determine the judgment result. The strategy fusion module 14 is used to determine the current game strategy based on the judgment result, the target supervised learning model in the current game robot, the target reinforcement learning model, the hand prediction rule and the strategy fusion rule, so as to realize game interaction according to the current game strategy.

[0073] In some specific embodiments, the first model training module 11 can be specifically used to: read real game logs from the online game server using a game log table; process the real game logs based on deduplication flow to determine the processed game logs; perform parallel parsing of the processed game logs based on a target state machine to determine the log parsing result; classify the processed game logs according to the log parsing result to determine the log classification result; the log classification result includes several model samples and several player win rate samples; the model samples include card-playing decisions and action decisions; perform game level analysis of player characters based on the player win rate samples to determine the level analysis result; and determine a player whitelist based on the level analysis result.

[0074] In some specific embodiments, the first model training module 11 can be specifically used to: construct a Python set set for the player whitelist in one go; sample the model samples based on the Python set set to determine the sampling results; determine the training dataset and the validation dataset based on the sampling results; analyze the proportion of decision categories of the samples in the training dataset to obtain the category proportion analysis results; perform downsampling processing on the training dataset based on the category proportion analysis results to determine the processing results; perform process-level shuffling on the processing results based on a multi-process data loader to determine the target training set; train the supervised learning model in the game robot using PyTorch based on the target training set until the training termination condition is met using the validation dataset to obtain the trained target supervised learning model; and export the target supervised learning model based on a preset protocol to obtain a model file.

[0075] In some specific embodiments, the second model training module 12 can be used to: acquire the initial reinforcement learning model in the game robot; create a shared memory buffer based on a distributed model-free control architecture; initialize the participant process pool and learner process pool based on the distributed model-free control architecture to obtain participant processes and learner processes; in the early stage of training, extract the last few card-playing information of each player character before winning a hand from the real game log based on the first information extraction index to obtain the initial training dataset; start the reinforcement learning model to perform self-game training in a simulated environment based on the initial training dataset to obtain the game results of the current self-game training round; collect self-game training data based on the participant processes and save the collected self-game training data to the shared memory buffer; the self-game training data includes the game results; and use the learner processes to retrieve data from the shared memory buffer. The process involves: reading the self-play training data to determine the reading result; using the learner process and the reading result to perform Markov decision process modeling to determine the modeling result; using the learner process and the modeling result to update the policy gradient and model parameters to determine the update result; continuing to train the reinforcement learning model in a simulated environment based on the update result until the mid-training stage, obtaining the model performance data of the current reinforcement learning model; determining a second information extraction index based on the model performance data; extracting the last few card-playing information of each player character before winning from the real game logs based on the second information extraction index, to obtain the updated target training dataset; and training the current reinforcement learning model in a simulated environment based on the target training dataset, the participant process, and the learner process until a preset loop termination condition is met, thus determining the trained target reinforcement learning model.

[0076] In some specific embodiments, the game judgment module 13 can be used to: obtain the current game status information of the target game from the online game server through the current game robot; record the current historical behavior information of each player character in the target game through the current game robot; wherein the current historical behavior information includes historical card playing information; record the information of the remaining cards outside the game in the target game through the current game robot to obtain the current remaining card information; determine the hand prediction result of each player character in the target game through the current game robot, and using the current historical behavior information, the current remaining card information, the current game status information, and the hand prediction rules; determine the winning probability through the current game robot and the hand prediction result; and determine whether the current target game belongs to a preset extreme game through the current game robot, the winning probability, a preset upper limit threshold, and a preset lower limit threshold, to obtain a first judgment result.

[0077] In some specific embodiments, the interactive device based on the card game robot can also be used to: determine whether the current target game is a preset extreme game by using the current game robot, current historical behavior information, current remaining card information and current game status information, and obtain a second judgment result.

[0078] In some specific embodiments, the strategy fusion module 14 can be used to: perform one-hot vector encoding on the current game robot, the hand prediction result, player role information, dealer information, current historical behavior information, current remaining card information, game stage information, and consecutive dealer information to obtain a basic state vector; determine a first strategy using the target reinforcement learning model in the current game robot and the basic state vector; the first strategy includes initial card playing decision probability information and initial action decision probability information; determine a second strategy using the target supervised learning model in the current game robot and the current game state information; determine current strategy fusion parameters using the current game robot, the first judgment result, and the second judgment result; and determine the current game strategy using the current game robot, the current strategy fusion parameters, the first strategy, and the second strategy.

[0079] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0080] Figure 7This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the interaction method of the card-based game robot disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0081] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0082] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0083] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the interactive method of the card-based game robot executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0084] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned interactive method for a card-based game robot. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0085] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0086] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0087] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0088] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An interaction method for a card-based gambling game robot, characterized in that, The game robot is a virtual game character controlled by an artificial intelligence algorithm, used to interact with the player character during game matches. The method includes: The real game logs of the online game server are analyzed, and the obtained samples and player whitelist are used to train the supervised learning model in the game bot using PyTorch to determine the trained target supervised learning model; wherein, the real game logs are the game logs of real player characters. The reinforcement learning model in the game robot is trained in a simulated environment using a distributed model-free control architecture and the real game logs to determine the trained target reinforcement learning model; the reinforcement learning model includes a card-playing model and an action decision model. In the target game, the hand prediction rules in the current game robot are used to determine whether the current target game belongs to a preset extreme game, and the current game status information is used to determine the judgment result; Based on the judgment result, the target supervised learning model, the target reinforcement learning model, the hand prediction rules, and the strategy fusion rules in the current game robot, the current game strategy is determined so as to realize game interaction according to the current game strategy; The step of determining whether the current target game match belongs to a preset extreme match by using the hand prediction rules in the current game robot and the current game state information includes: The current game bot obtains the current game status information of the target game from the online game server. The game bot records the current historical behavior information of each player character in the target game; wherein, the current historical behavior information includes historical card playing information; The remaining card information is obtained by recording the information of the remaining cards outside the target game session using the current game robot. Using the current game bot, and utilizing current historical behavior information, current remaining card information, current game state information, and hand prediction rules, the hand prediction results for each player character in the target game are determined; The probability of winning is determined by the current game robot and the hand prediction results. The first judgment result is obtained by determining whether the current target game match belongs to a preset extreme match based on the current game robot, the winning probability, the preset upper limit threshold, and the preset lower limit threshold.

2. The interaction method for a card-based game robot according to claim 1, characterized in that, The analysis of real game logs from the online game server includes: Use the game log table to read the actual game logs from the online game server; The actual game logs are processed based on the deduplication flow to determine the processed game logs; The processed game log is parsed in parallel based on the target state machine to determine the log parsing result; The processed game logs are classified according to the log parsing results to determine the log classification results; the log classification results include several model samples and several player win rate samples; the model samples include card-playing decisions and action decisions; Based on the player win rate sample, analyze the player character's game level to determine the level analysis results; The player whitelist is determined based on the results of the horizontal analysis.

3. The interaction method for a card-based game robot according to claim 2, characterized in that, The step of using the obtained samples and player whitelist to train the supervised learning model in the game bot using PyTorch to determine the trained target supervised learning model includes: Construct the player whitelist into a Python set all at once; The model samples are sampled based on the Python set to determine the sampling results; The training and validation datasets are determined based on the sampling results; The proportion of decision categories in the samples of the training dataset is analyzed to obtain the category proportion analysis results; The training dataset is downsampled based on the category proportion analysis results to determine the processing results; The processing results are shuffled at the process level using a multi-process data loader to determine the target training set. The supervised learning model in the game robot is trained using PyTorch based on the target training set until the training termination condition is met using the validation dataset, thus obtaining the trained target supervised learning model. The target supervised learning model is exported based on a preset protocol to obtain a model file.

4. The interaction method for a card-based game robot according to claim 1, characterized in that, The process of training the reinforcement learning model in the game robot in a simulated game environment based on the distributed model-free control architecture and the real game logs to determine the trained target reinforcement learning model includes: Obtain the initial reinforcement learning model in the game robot; A shared memory buffer is created based on a distributed model-free control architecture; The participant process pool and learner process pool are initialized based on the distributed model-free control architecture to obtain participant processes and learner processes. In the early stage of training, based on the first information extraction index, the last few card-playing information of each player character before winning the hand in the real game logs is extracted to obtain the initial training dataset. The reinforcement learning model is launched based on the initial training dataset to conduct self-game training in a simulated environment in order to obtain the game results of the current self-game training round. The self-play training data is collected based on the participant process, and the collected self-play training data is saved to the shared memory buffer; the self-play training data includes the game results; The learner process reads the self-play training data from the shared memory buffer to determine the reading result; The learner process and the reading results are used to model a Markov decision process to determine the modeling results. The policy gradient and model parameters are updated using the learner process and the modeling results to determine the update results; Based on the updated results, the reinforcement learning model continues to be trained in a simulated game environment until the mid-training stage, at which point the model performance data of the current reinforcement learning model is obtained. The second information extraction index is determined based on the model performance data; Based on the second information extraction index, the last few card-playing information of each player character before winning the hand in the real game log is extracted from the real game log to obtain the updated target training dataset. The current reinforcement learning model is trained in a simulated game environment based on the target training dataset, the participant process, and the learner process until a preset loop termination condition is met, at which point the trained target reinforcement learning model is determined.

5. The interaction method for a card-based game robot according to claim 1, characterized in that, Also includes: The second judgment result is obtained by determining whether the current target game game belongs to the preset extreme game by using the current game robot, current historical behavior information, current remaining card information and current game status information.

6. The interaction method for a card-based game robot according to claim 5, characterized in that, The process of determining the current game strategy based on the judgment result, the target supervised learning model in the current game robot, the target reinforcement learning model, the hand prediction rules, and the strategy fusion rules includes: The basic state vector is obtained by performing single-hot vector encoding on the current game robot, the hand prediction result, player role information, dealer information, current historical behavior information, current remaining card information, game stage information, and consecutive dealer information; A first strategy is determined using the target reinforcement learning model and the basic state vector in the current game robot; the first strategy includes initial card-playing decision probability information and initial action decision probability information. The second strategy is determined by using the target-supervised learning model in the current game bot and by utilizing the current game state information; The current strategy fusion parameters are determined by the current game robot, the first judgment result, and the second judgment result. The current game strategy is determined by the current game robot, the current strategy fusion parameters, the first strategy, and the second strategy.

7. An interactive device for a card-based gaming robot, characterized in that, The game robot is a virtual game character controlled by an artificial intelligence algorithm, used to interact with the player character during game matches. The device includes: The first model training module is used to parse the real game logs of the online game server, and use the obtained samples and player whitelist to train the supervised learning model in the game robot using PyTorch, so as to determine the trained target supervised learning model; wherein, the real game logs are the game logs of real player characters. The second model training module is used to train the reinforcement learning model in the game robot in a simulated environment based on the distributed model-free control architecture and the real game logs, so as to determine the trained target reinforcement learning model; the reinforcement learning model includes a card playing model and an action decision model. The game judgment module is used to determine whether the current target game game belongs to a preset extreme game game by using the hand prediction rules in the current game robot and the current game state information in the target game game, so as to determine the judgment result; The strategy fusion module is used to determine the current game strategy based on the judgment result, the target supervised learning model, the target reinforcement learning model, the hand prediction rule, and the strategy fusion rule in the current game robot, so as to realize game interaction according to the current game strategy; The game judgment module is configured to: obtain the current game status information of the target game from the online game server through the current game robot; record the current historical behavior information of each player character in the target game through the current game robot; wherein the current historical behavior information includes historical card playing information; record the information of the remaining cards outside the game through the current game robot to obtain the current remaining card information; determine the hand prediction result of each player character in the target game through the current game robot, and using the current historical behavior information, the current remaining card information, the current game status information, and the hand prediction rules; determine the winning probability through the current game robot and the hand prediction result; and determine whether the current target game belongs to a preset extreme game through the current game robot, the winning probability, a preset upper threshold, and a preset lower threshold, to obtain a first judgment result.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the interaction method of the card-based game robot as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the interaction method of the card-based game robot as described in any one of claims 1 to 6.

Images