AI mahjong auxiliary guiding method

Abstract

The invention discloses an AI mahjong auxiliary guiding method. The method comprises the following steps of S1, obtaining mahjong score data; s2, training an AI model according to the sign data; s3, obtaining observation value data from the game logic; s4, driving the AI model through the observed value data to obtain a reasoning result; and S5, providing guide information on the game interface for the player according to a reasoning result. The method has the advantages that the AI model is trained according to the card score data through the near-end strategy optimization algorithm, when a player plays a game, the system can collect and analyze the current observed value data of the player in real time, the data are input into the trained AI model, the model is driven to conduct reasoning calculation, the optimal decision scheme under the current situation is analyzed, and the optimal decision scheme under the current situation is obtained. Therefore, players are assisted in making more intelligent decisions, and the game winning rate is increased.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an AI mahjong auxiliary guidance method. Background Art

[0002] As a game that combines strategy, skills and luck, Japanese Mahjong has long been loved and sought after by players around the world. Its unique rule system makes each game full of variables and challenges, requiring players to have a high level of analytical ability, decision-making speed and deep experience accumulation. With the rapid development of artificial intelligence (AI) technology, especially the breakthrough progress in deep learning, big data processing and reinforcement learning, the application of AI in various fields is becoming more and more extensive and in-depth. In the field of Japanese Mahjong games, the introduction of AI technology has brought unprecedented changes and innovations to the game.

[0003] In the Japanese Mahjong game, the diversity of strategies is one of the keys to winning. However, many existing Japanese Mahjong game AIs are limited by their programmed fixed algorithms and strategy frameworks, which makes them unable to cope with the diverse game situations. These AIs usually make decisions based on preset card evaluations and simple probability statistics, and lack a deep understanding of the complex psychology and tactical changes of human players. For example, when three players compete for the same good hand in the game, or someone is obviously making a big hand, the existing AIs are often unable to flexibly adjust their strategies, but continue to act according to the established algorithms, which may cause them to make wrong decisions at certain critical moments, such as blindly following the play, giving up the opportunity to win, or exposing their intentions too early; in addition, the existing AIs can often only handle some common card types and combinations, and are helpless with some special card types and complex changes in the game, which limits the depth of AI's performance in the game. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an AI mahjong auxiliary guidance method.

[0005] The purpose of the present invention is achieved through the following technical solutions: an AI mahjong auxiliary guidance method, comprising the following steps:

[0006] S1: Get card score data;

[0007] S2: Train the AI ​​model based on the card data;

[0008] S3: Get observation data from game logic;

[0009] S4: Drive the AI ​​model through observation data to obtain inference results;

[0010] S5: Provide guidance information on the game interface to the player based on the inference results.

[0011] Preferably, in step S2, the AI ​​model is trained by a proximal strategy optimization algorithm.

[0012] Preferably, in step S2, the steps of training the AI ​​model by the proximal strategy optimization algorithm are as follows:

[0013] S21: Initialize the policy network and value network, and create an experience replay buffer;

[0014] S22: Interact with the environment according to the current strategy, collect samples of state, action, reward, and next state, and store them in the experience replay buffer;

[0015] S23: Estimate the value of each state using the value network and calculate the advantage function;

[0016] S24: Draw a batch of samples from the experience replay buffer and use these samples to update the policy network and value network;

[0017] S25: Repeat steps S22 to S24 until the convergence condition or the number of training steps is reached.

[0018] Preferably, in step S24, the policy network parameters are updated by minimizing an objective function, and the objective function is:

[0019]

[0020] Among them, θ represents the expectation of the collected samples, represents the estimated advantage value, ∈ is a hyperparameter with a value range of 0.1-0.9, which is used to limit the strategy update amplitude, r t (θ) represents the probability ratio of the new and old strategies, and clip represents if r t If (θ) is less than 1-∈ or greater than 1+∈, the value will be truncated to 1-∈ or 1+∈;

[0021]

[0022] Among them, π θ (a t |s t ) represents the new strategy π θ In s t Execute action a in state t The probability of Indicates the old policy.

[0023] Preferably, in step S24, the value network parameters are updated by a mean square error loss function.

[0024] Preferably, the observation value data includes player hand information, player side-reveal information, card face information, played card information and riichi information.

[0025] The present invention has the following advantages: the present invention trains the AI ​​model according to the card data through the proximal strategy optimization algorithm. When the player plays the game, the system collects and analyzes the player's current observation value data in real time. The data is input into the trained AI model to drive the model to perform reasoning calculations and analyze the optimal decision-making plan under the current situation, that is, whether to eat, touch, connect or stand upright, etc., to assist the player to make more wise decisions and improve the game winning rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a structural diagram of the AI ​​mahjong assisted guidance method process. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0028] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0030] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0031] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use, or the positions or positional relationships commonly understood by those skilled in the art, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0032] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0033] In this embodiment, if Figure 1 As shown, an AI mahjong auxiliary guidance method is characterized by comprising the following steps:

[0034] S1: Get card score data;

[0035] S2: Train the AI ​​model based on the card data;

[0036] S3: Obtain observation value data from the game logic; preferably, the observation value data includes player hand information, player side-reveal information, card face information, card played information and standing card information.

[0037] S4: Drive the AI ​​model through observation data to obtain inference results;

[0038] S5: Provide players with guidance information on the game interface based on the inference results. The AI ​​model is trained based on the card data through the proximal strategy optimization algorithm. When the player is playing, the system will collect and analyze the player's current observation value data in real time. The data is input into the trained AI model, driving the model to perform inference calculations and analyze the optimal decision-making plan under the current situation, that is, whether to eat, touch, connect or stand up, etc., to assist players in making more intelligent decisions and improve the game winning rate.

[0039] Furthermore, in step S2, the AI ​​model is trained by a proximal policy optimization algorithm. Specifically, the proximal policy optimization algorithm is a reinforcement learning algorithm based on policy gradient, which optimizes the policy to maximize the cumulative reward. In the proximal policy optimization, the policy is represented by a neural network, which takes the current state as input and outputs the probability distribution of each action.

[0040] Furthermore, in step S2, the steps of training the AI ​​model through the proximal strategy optimization algorithm are as follows:

[0041] S21: Initialize the policy network and value network, and create an experience replay buffer;

[0042] S22: Interact with the environment according to the current strategy, collect samples of state, action, reward, and next state, and store them in the experience replay buffer;

[0043] S23: Estimate the value of each state using the value network and calculate the advantage function;

[0044] S24: Draw a batch of samples from the experience replay buffer and use these samples to update the policy network and value network;

[0045] S25: Repeat steps S22 to S24 until the convergence condition or the number of training steps is reached. Specifically, the goal of updating the policy network is to maximize the expected return of the policy while limiting the range of policy changes to ensure the stability of the update.

[0046] In this embodiment, in step S24, the policy network parameters are updated by minimizing the objective function, and the objective function is:

[0047]

[0048] Among them, θ represents the expectation of the collected samples, represents the estimated advantage value, ∈ is a hyperparameter with a value range of 0.1-0.9, which is used to limit the strategy update amplitude, r t (θ) represents the probability ratio of the new and old strategies, and clip represents if r t If (θ) is less than 1-∈ or greater than 1+∈, the value will be truncated to 1-∈ or 1+∈.

[0049]

[0050] Among them, π θ (a t |s t ) represents the new strategy π θ In s t Execute action a in state t The probability of In step S24, the value network parameters are updated by the mean square error loss function.

[0051] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An AI mahjong auxiliary guidance method, characterized in that: The following steps are involved: S1: Get card score data; S2: Train the AI ​​model based on the card data; S3: Get observation data from game logic; S4: Drive the AI ​​model through observation data to obtain inference results; S5: Provide guidance information on the game interface to the player based on the inference results.

2. The AI ​​mahjong auxiliary guidance method according to claim 1, characterized in that: In step S2, the AI ​​model is trained using a proximal strategy optimization algorithm.

3. The AI ​​mahjong auxiliary guidance method according to claim 2, characterized in that: In step S2, the steps of training the AI ​​model by the proximal strategy optimization algorithm are as follows: S21: Initialize the policy network and value network, and create an experience replay buffer; S22: Interact with the environment according to the current strategy, collect samples of state, action, reward, and next state, and store them in the experience replay buffer; S23: Estimate the value of each state using the value network and calculate the advantage function; S24: Draw a batch of samples from the experience replay buffer and use these samples to update the policy network and value network; S25: Repeat steps S22 to S24 until the convergence condition or the number of training steps is reached.

4. The AI ​​mahjong auxiliary guidance method according to claim 3, characterized in that: In step S24, the policy network parameters are updated by minimizing the objective function, where the objective function is: Among them, θ represents the expectation of the collected samples, represents the estimated advantage value, ∈ is a hyperparameter with a value range of 0.1-0.9, which is used to limit the strategy update amplitude, r t (θ) represents the probability ratio of the new and old strategies, and clip represents if r t If (θ) is less than 1-∈ or greater than 1+∈, the value will be truncated to 1-∈ or 1+∈; Among them, π θ (a t |s t ) represents the new strategy π θ In s t Execute action a in state t The probability of Indicates the old policy.

5. The AI ​​mahjong auxiliary guidance method according to claim 4, characterized in that: In step S24, the value network parameters are updated by the mean square error loss function.

6. The AI ​​mahjong auxiliary guidance method according to claim 5, characterized in that: The observed value data includes player hand information, player side-reveal information, card face information, played card information and riichi information.

Images