Data processing method and device for asymmetric game, electronic equipment and storage medium

Abstract

The application provides a data processing method and device for an asymmetric game, an electronic device and a storage medium. A non-player character is trained through a man-machine training game, and a trained non-player character of all ranks is obtained. After modifying game parameters, the trained non-player character is used to simulate a game, and game data change information is determined according to game data after the parameters are modified and game data before the parameters are modified. The balance state of the current game is determined based on the game data change information, and whether to modify the game parameters is determined based on the balance state of the game. The application achieves the dual goal of modifying the game parameters while not affecting the overall balance of the game, thereby optimizing the balance of the game.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a data processing method and device for asymmetric games, an electronic device and a storage medium. BACKGROUND

[0002] Asymmetric competitive games are a type of multiplayer game in which different players play different roles, have different skills, goals and abilities. The goals and abilities of these players are different, so the balance and difficulty of the game often need to be carefully designed to ensure.

[0003] Machine learning has been used as a solution to automatically optimize the balance of asymmetric competitive games in the prior art. By collecting game data after the game goes online, using machine learning techniques to analyze the game data, predicting balance problems in the game, and proposing corresponding optimization solutions.

[0004] However, the quality of the game data in the prior art has a great influence on the accuracy and reliability of the analysis results. However, there may be noise and errors in the data collection process, and the preferences of players can also cause data drift. In addition, the algorithm complexity is high, so it is difficult to optimize the balance of the game in practice. SUMMARY

[0005] The purpose of the present application is to solve the problem of optimizing the balance of the game in practice in the prior art by providing a data processing method and device for asymmetric games, an electronic device and a storage medium.

[0006] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows:

[0007] In a first aspect, the present application provides a data processing method for asymmetric games, comprising:

[0008] Based on the game results of the non-player character in the training game of the asymmetric game, iteratively update the behavior Q value table of the non-player character until the training of the non-player character is obtained when the iteration termination condition is met. The behavior Q value table is used to represent the Q value corresponding to the state action of the non-player character;

[0009] Obtain the first game data of the trained non-player character in the first simulation game of the asymmetric game; the first simulation game includes the first game parameter before modification;

[0010] Obtain the second game data of the trained non-player character in the second simulation game of the asymmetric game, the second simulation game includes the second game parameter after modifying the first game parameter;

[0011] determine game data change information of the non-player character according to the second game data and the first game data;

[0012] determine a game state of the asymmetric game based on the game data change information, to determine whether to modify the second game parameter based on the game state; the game state includes a balanced state and an unbalanced state.

[0013] In a second aspect, the present application provides a data processing apparatus of an asymmetric game, the apparatus comprising:

[0014] a training module configured to iteratively update a behavior Q-value table of the non-player character based on game results of the non-player character in training games of the asymmetric game, until a trained non-player character is obtained when an iteration termination condition is met; the behavior Q-value table is used to represent Q-values corresponding to state actions of the non-player character;

[0015] a first obtaining module configured to obtain first game data of the trained non-player character in a first simulated game of the asymmetric game; the first simulated game includes a first game parameter before modification;

[0016] a second obtaining module configured to obtain second game data of the trained non-player character in a second simulated game of the asymmetric game; the second simulated game includes a second game parameter after modification of the first game parameter;

[0017] a change determining module configured to determine game data change information of the non-player character according to the second game data and the first game data;

[0018] a state determining module configured to determine a game state of the asymmetric game based on the game data change information, to determine whether to modify the second game parameter based on the game state; the game state includes a balanced state and an unbalanced state.

[0019] In a third aspect, the embodiments of the present application further provide an electronic device, comprising a processor, a storage medium and a bus, the storage medium storing machine readable instructions executable by the processor, when the electronic device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine readable instructions to perform the steps of the data processing method of the asymmetric game according to any one of the first aspect.

[0020] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program. When the computer program is run by a processor, the steps of the data processing method of the asymmetric game according to any one of the first aspect are executed.

[0021] The beneficial effects of the present application are as follows: the non-player character is trained in the training game, and the behavior Q value table of the non-player character is updated according to the game operation result of the non-player character in the training game, so that the non-player character is obtained, which is full of all roles and all levels and is closer to the level of a real human player. After the non-player character is obtained, the training non-player character is used to simulate the game, so that the game process of different players can be simulated, a large amount of real and objective game data can be obtained in a short time, and problems and defects in the game can be quickly found in a short time, so that the game can be repaired and optimized in time. After a large amount of game data is obtained by using the non-player character to simulate the game, the balance of the game can be quickly measured based on the game data and the data before the modification parameters, so that the efficiency and effect of the game balance optimization are improved.

[0022] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0024] Figure 1 A schematic diagram of an application scenario provided by the embodiments of the present application is shown;

[0025] Figure 2 A flowchart of a data processing method of an asymmetric game provided by the embodiments of the present application is shown;

[0026] Figure 3 A flowchart of updating a behavior Q value table provided by the embodiments of the present application is shown;

[0027] Figure 4 A flowchart of determining a to-be-executed action provided by the embodiments of the present application is shown;

[0028] Figure 5 A flowchart of determining an updated Q value of a to-be-executed action provided by the embodiments of the present application is shown;

[0029] Figure 6 A flowchart of determining whether a behavior Q-value table is effective is shown in the embodiments of the present application.

[0030] Figure 7 A flowchart of determining game data change information is shown in the embodiments of the present application.

[0031] Figure 8 A flowchart of determining game data change information is shown in the embodiments of the present application.

[0032] Figure 9 A structural schematic diagram of a data processing apparatus of an asymmetric game is shown in the embodiments of the present application.

[0033] Figure 10 A structural schematic diagram of an electronic device is shown in the embodiments of the present application. DETAILED DESCRIPTION

[0034] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the accompanying drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0035] It should be noted that the term “comprising” will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0036] Asymmetric competitive games are a type of multi-player games, in which different players play different roles, have different skills, goals and abilities. The goals and abilities of these players are different, so the balance and difficulty of the game often need to be carefully designed to ensure.

[0037] In asymmetric competitive games, there are complex and subtle relationships between each role, rules, and scene design and other factors. Only by ensuring the coordination and balance of various parameters can players enjoy a better experience.

[0038] However, it is not easy to maintain balance in practice, because in asymmetric competitive games, each player has his own unique style and strategy, making data collection and analysis difficult, and there are many elements in the game, different role combinations can also cause data sparsity, and the product iteration speed is fast, the newly added functions will also affect the overall balance, and the player's demand is obviously different, so the balance optimization of asymmetric competitive games has become a problem to be solved.

[0039] The existing technology for balancing asymmetric competitive games includes machine learning, data mining and user feedback.

[0040] Among them, machine learning uses machine learning technology to analyze game data, predict balance problems in the game, and propose corresponding optimization solutions.

[0041] However, the machine learning algorithm has high complexity and needs reliable data support, but the game data has low data quality due to the influence of players' personal preferences and noise in the collection process, so using machine learning to optimize balance is still insufficient.

[0042] Data mining refers to the game planner using data mining technology to mine the rules and patterns in the game data, find the balance problems in the game, and optimize the balance through version fine-tuning.

[0043] However, this method depends on the subjective experience of game planners, and if the planners cannot fully understand the game mechanism and players' needs, they may incorrectly modify some values, resulting in deviations in the final balance optimization results.

[0044] User feedback refers to collecting user feedback and opinions through a questionnaire system, sorting out data to analyze users' behavior patterns and needs, and finding balance problems in the game. Game planners propose corresponding optimization solutions based on the problems found.

[0045] However, user feedback information is highly subjective, and the quality and quantity of user feedback may be problematic, so it can also cause deviations in game balance optimization results.

[0046] In summary, the existing technology for balancing asymmetric competitive games has problems such as low data quality, high algorithm complexity, and excessive subjectivity of user feedback and game planners.

[0047] Based on the above problems, the application provides a data processing method of asymmetric game. After the game version is iteratively updated, the robot is trained to learn different roles to play the game, so as to ensure that the robot has a reasonable win rate at different levels of the role and abnormality does not occur due to different operations of the player. After the training is completed, the robot is used to simulate the game, and the influence of the game version iterative update on the game balance is verified and optimized based on the change information of the game data.

[0048] Firstly, the basic process of optimizing balance in the application is described. After a new role or a new game map is released, the game version needs to be iteratively updated. After the game version is iteratively updated, the balance of the game can be ensured by adjusting some parameters in the game. When the same role at the same level has no obvious change in the game data before and after the parameter modification, it is considered that the game is in a balanced state. Otherwise, it is considered that the balance of the game is broken, and the game parameters need to be adjusted again.

[0049] As shown in Figure 1 , it is a brief flowchart of the method of the application. After the game version is iteratively updated, the training game between the non-player character and the real player can be carried out on the electronic device based on the method of the application, so as to train the non-player character of all levels and all roles. The non-player character can be a game robot. Then, the game data of different combinations is obtained by simulating the game with the robot, so as to determine the balance state of the current game based on the game data, and adjust the game parameters to realize the balance optimization.

[0050] Compared with the prior art, the algorithm of the application has low complexity and comprehensive and objective data, can provide more objective and real game evaluation results, and can save the game simulation time through the game robot, save a large amount of test cost, and improve the test efficiency and effect.

[0051] Next, the data processing method of asymmetric game of the application will be described in combination with specific embodiments. The execution subject of the method can be an electronic device, as shown in Figure 2 , the method comprises the following steps.

[0052] S201: based on the game result of the non-player character in the training game of the asymmetric game, iteratively updating the behavior Q value table of the non-player character until the iteration termination condition is met to obtain the trained non-player character; the behavior Q value table is used to represent the Q value corresponding to the state action of the non-player character.

[0053] Optionally, the non-player character can be a program script running on the electronic device to simulate the real operation of the player. The tester can create multiple processes on the electronic device, and each process runs a non-player character of a different role.

[0054] As another possible implementation, the tester can also create multiple processes on different electronic devices respectively to run different non-player characters.

[0055] Optionally, the electronic device can create or participate in different training games, and use non-player characters of different roles and different levels to learn simulation games in the training games, which include non-player characters and real human players.

[0056] In this way, the non-player characters of the same level and the same role share the same storage space, and the game operation results generated in the training game can be stored in the storage space.

[0057] Optionally, the game operation result can be game data generated by the non-player character in the training game, such as an action performed by the non-player character in a certain state in the game scene and a result caused by the action.

[0058] In this way, the state of the non-player character can include the position of the non-player character in the game scene, the life information of the non-player character, and the prop information currently possessed by the non-player character, etc. The life information can include blood volume and skill cooling time, and the prop information can include the prop currently used by the non-player character, etc.

[0059] The action of the non-player character can be a game operation that can be performed in the current state of the non-player character, such as up, down, left, right, attack, and skill release in a certain map position.

[0060] Optionally, for each role of different levels, the non-player character can initialize a behavior Q value table, and each value in the initial behavior Q value table can be 0 or a random number. In the process of the training game, the initial behavior Q value table can be updated based on the game operation result of the non-player character to obtain the behavior Q value table of the role of the current level of the non-player character in the training game.

[0061] As shown in Table 1, an example of a behavior Q value table is given in this application, wherein S1 represents a state, and the numbers below each action represent the Q value of the action in the state of S1.

[0062] Table 1: Behavior Q value example

[0063]

[0064] Wherein, the Q value can represent the influence degree of the action on the result in the S1 state, the greater the Q value, the more likely the action in the S1 state leads to the game victory, and the smaller the Q value, the more likely the action in the S1 state leads to the game failure.

[0065] It is worth noting that in the training game, the non-player characters can be arranged to fight with the real human players, and the proportion of the non-player characters in the training game can not exceed a preset proportion of the total number of games, for example, 10%, so that the non-player characters can learn the game strategies of different characters in the near-real game.

[0066] In this application, the non-player characters can iteratively perform multiple training games, iteratively update the behavior Q value table, and save the behavior Q value table of the non-player characters obtained by the last training when the behavior Q value table meets the iteration termination condition.

[0067] S202: Obtain first game data of the trained non-player characters in the first simulation game of the asymmetric game; the first simulation game includes the first game parameters before the parameter modification.

[0068] After obtaining the trained non-player characters of all ranks and all characters, the simulation games of different character combinations can be created, and the game robots can be triggered to perform game operations in the simulation games.

[0069] Wherein, all participants in the first simulation game can be non-player characters, and the characters and ranks of the non-player characters can be randomly selected. The electronic device can select different permutations and combinations to perform multiple simulation games to obtain more comprehensive game data.

[0070] Optionally, the first game data represents the game data before the game parameter modification, and the first game parameters can include: skill cooling time of characters in the game, attack damage value of characters, and other configuration information that can affect the strength of the characters.

[0071] S203: Obtain second game data of the trained non-player characters in the second simulation game of the asymmetric game, and the second simulation game includes the second game parameters after the modification of the first game parameters.

[0072] After obtaining the first simulation game data, the game parameters of the asymmetric game can be modified, the first game parameters are modified into the second game parameters, for example, adjusting the skill cooling time of characters in the game, the attack damage value of characters, and the like.

[0073] Wherein, all participants in the second simulation game are non-player characters, and the second game data represents the game data after the game parameter modification.

[0074] It should be noted that the strategy of the non-player character for selecting the next execution action in the first simulation game and the second simulation game can be the same as the strategy of the non-player character for selecting the next execution action in the training game.

[0075] S204: determining game data change information of the non-player character according to the second game data and the first game data.

[0076] When the non-player character performs the simulation game, the game operation result of the non-player character in the simulation game can be transmitted to the database in real time, and the database can automatically perform data statistics to obtain the game data of each character in different ratings in the simulation game.

[0077] Before performing the second simulation game, the game parameters can be modified by the developer, and after the second simulation game is completed, the game data change information of the non-player character can be determined according to the second game data of the second simulation game and the first game data of the first simulation game of the non-player character. If the game data changes greatly before and after the modification, it indicates that the parameter modification affects the balance of the game, and at this time, the game parameter modification can be performed again.

[0078] The game data change information can include the win rate change and the ranking change of the same character in the same rating before and after the game parameter modification.

[0079] S205: determining a game state of the asymmetric game based on the game data change information, to determine whether to modify the second game parameter based on the game state; the game state includes a balanced state and an unbalanced state.

[0080] When the game is in the balanced state, the win rate difference of each participant is less than a preset threshold in a game in which n participants with the same level use various characters of the same rating to participate in the game under the premise of fully understanding the game rules.

[0081] When the game is in the unbalanced state, the win rate difference of at least one participant is greater than a preset threshold in a game in which n participants with the same level use various characters of the same rating to participate in the game under the premise of fully understanding the game rules.

[0082] Optionally, according to the game data change information, the developer can know the influence of the parameter modification on the game character, determine the game state of the current asymmetric game, continue to modify the game parameter when the asymmetric game is in the unbalanced state, and perform the above steps S203-S204 again until the game is in the balanced state.

[0083] As a possible implementation, the game parameters of a game character can be modified based on the game data change information of the game character in all simulated matches, to ensure that the character does not affect the balance of the game.

[0084] It is worth noting that the above steps S202-S205 can be iterated multiple times to ultimately achieve the dual goal of modifying game parameters while not affecting the overall balance of the game, thereby optimizing the balance of the game.

[0085] In the embodiments of the present application, the non-player character is trained through the training match, and the non-player character of all ranks and all characters is obtained. After modifying the game parameters, the trained non-player character is used to perform the first simulated match to obtain the first match data, then the game parameters are modified and the second simulated match is performed to obtain the second match data, and finally the match data change information is determined according to the first match data and the second match data, so as to determine the game state of the asymmetric game based on the match data change information, and determine whether to modify the game parameters. The present application realizes the dual goal of modifying the game parameters while not affecting the overall balance of the game, thereby optimizing the balance of the game.

[0086] The non-player character is trained in the training match, and the behavior Q value table of the non-player character is obtained according to the game operation result of the non-player character in the training match. The non-player character of all ranks and all characters can be obtained, which is closer to the level of a real human player, and the behavior Q value tables of the non-player characters between different versions have similarities, so they can be reused to some extent.

[0087] By using the trained non-player character to perform the simulated match, not only the game process of different player levels can be simulated, but also a large amount of real and objective match data can be obtained in a short time, and problems and defects existing in the game can be quickly found in a short time, so that timely repair and optimization can be facilitated. After a large amount of match data is obtained by using the non-player character to perform the simulated match, the balance of the game can be quickly measured based on the match data and the data before the modification of the parameters, thereby improving the efficiency and effect of the balance optimization of the game.

[0088] Next, the process of obtaining the behavior Q value table of the non-player character according to the game operation result of the non-player character in each training match is described. Figure 3 As shown in FIG. 7, the step S201 includes:

[0089] S301: determining the to-be-executed action of the non-player character according to the current state of the non-player character in the current training match and action source information, wherein the action source information includes: a current behavior Q value table or an action selection strategy, and the current behavior Q value table is used to represent the Q value corresponding to the state-action of the non-player character in the current training match.

[0090] Optionally, the non-player character can select one of the plurality of actions in the current state as the to-be-executed action based on a current behavior Q-value table or an action selection policy.

[0091] The current behavior Q-value table includes, in the current training game, Q-values of each action corresponding to each state.

[0092] The action selection policy can be that one of the actions in the current state is randomly selected as the to-be-executed action with a certain probability.

[0093] It should be noted that, in the training process, in order to more comprehensively explore new states and actions, the non-player character can randomly determine the action source information with a certain probability, that is, the non-player character can randomly select the to-be-executed action of the non-player character according to the current behavior Q-value table or according to the action selection policy.

[0094] S302: Trigger the non-player character to execute the to-be-executed action.

[0095] In the training game, after the electronic device determines the to-be-executed action of the non-player character, the non-player character can execute the to-be-executed action and obtain the next state after executing the to-be-executed action.

[0096] S303: According to the result of the non-player character executing the to-be-executed action, determine the updated Q-value corresponding to the to-be-executed action, and update the current behavior Q-value table according to the updated Q-value corresponding to the to-be-executed action.

[0097] Optionally, the result of the non-player character executing the to-be-executed action can be the state of the non-player character after executing the to-be-executed action, including the position of the character in the game map, the survival state of the character, the blood volume of the character, and the skill cooling time of the character.

[0098] According to the result of the non-player character executing the to-be-executed action, the Q-value of the to-be-executed action can be determined. For example, referring to Table 1, it is assumed that the survival state of the character changes from survival to non-survival after the non-player character executes the action of "moving up" in the current state, which indicates that the Q-value of the to-be-executed action in the current state is low. At this time, the updated Q-value of the to-be-executed action can be determined based on the current state and the result of executing the to-be-executed action, and the updated Q-value is taken as the Q-value of the action of "moving up" in the S1 state of the current behavior Q-value table.

[0099] S304: Obtain the updated current behavior Q-value table when the current training game ends, and obtain the current behavior Q-value table of the next training game according to the updated current behavior Q-value table.

[0100] After each action is executed in the training game, the S303 step is executed to update the current behavior Q value table. At the end of the current training game, the current behavior Q value table of each non-player character in the training game is obtained.

[0101] It should be noted that the current behavior Q value table of the non-player character refers to the current behavior Q value table of the character used by the non-player character at the current rating.

[0102] It should be understood that the results caused by the same action in the same state can have randomness, and therefore the behavior Q value table of the character at the rating can be obtained by iterating the current behavior Q value table through multiple training of the same character at the same rating in the present application.

[0103] For example, after the first training game ends, the behavior Q value table obtained in the first training game can be used as the current behavior Q value table in the second training game, and after the second training game ends, the updated behavior Q value table obtained can be used as the current behavior Q value table in the third training game.

[0104] S305: Determine whether the current behavior Q value table meets the iteration termination condition. If yes, the updated current behavior Q value table obtained in the last training game is used as the behavior Q value table of the non-player character.

[0105] Optionally, the current behavior Q value table meeting the iteration termination condition can be that the Q value in the behavior Q value table meets a preset convergence condition, or the number of training games reaches a preset iteration number. At this time, the updated current behavior Q value table obtained in the last training game is used as the behavior Q value table of the non-player character, and the behavior Q value table is stored in the corresponding storage space of the rating of the character currently used by the non-player character.

[0106] The preset convergence condition can be that the change of the Q value is less than a preset change value after multiple iterations and updates.

[0107] In the embodiments of the present application, by determining and executing the action based on the behavior Q value table or the action selection strategy in the training game, sufficient exploration of each action in each state can be achieved, thereby achieving a more comprehensive training effect. Through continuous iteration and optimization of the training game, the authenticity and reliability of the data can be improved, and the problem of incomplete data caused by few iterations can be avoided.

[0108] The following is a further description of the above determination of the to-be-executed action of the non-player character based on the current state of the non-player character in the current training game and the action source information, as shown in Figure 4 The S301 step includes:

[0109] S401: If the action source information is the current behavior Q value table, read the Q value of each action in the current state from the current behavior Q value table.

[0110] S402: Determine the action to be executed by the non-player character according to the Q value of each action in the current state.

[0111] As a possible implementation, if the non-player character determines the action to be executed according to the current behavior Q value table, the non-player character can read the Q value of each action in the current state from the current behavior Q value table, and select the optimal action in the current state as the action to be executed in each state, so that the robot learns the game strategy of different characters.

[0112] For example, the action with the highest Q value in the current state in the behavior Q value table can be selected as the action to be executed.

[0113] After determining the action to be executed, the selected action to be executed can be executed, and the Q value of the action to be executed in the current state in the behavior Q value table can be updated.

[0114] It should be noted that if the selection of the action to be executed is only based on the above S401-S402 steps, it is possible that only one action with the highest Q value is selected each time, resulting in incomplete exploration. Therefore, as another possible implementation, the present application can also determine the action to be executed by the non-player character based on the action selection strategy. The selection of the action to be executed based on the action selection strategy and the selection of the action to be executed based on the behavior Q value table can be alternately or randomly performed, which is not limited by the present application.

[0115] When the action source information is the action selection strategy, the above S301 step includes:

[0116] If the action source information is the action selection strategy, determine the action to be executed by the non-player character based on the action selection strategy.

[0117] Optionally, the action selection strategy can be an ε-greedy strategy, that is, an action is randomly selected as the action to be executed among the actions corresponding to the current state under a certain probability, so as to explore new states and actions.

[0118] After determining the action to be executed, the updated Q value corresponding to the action to be executed can be determined according to the result of the non-player character executing the action to be executed, such as Figure 5 The above S303 step includes:

[0119] S501: Determine the next state of the non-player character and the reward value of the next state according to the result of the non-player character executing the action to be executed.

[0120] After the non-player character performs the to-be-performed action, the position after the non-player character performs, the blood volume of the non-player character, the survival state, and the skill cooling condition of the non-player character can be obtained, that is, the next state of the robot and each executable action of the next state are obtained.

[0121] Optionally, the reward value of the next state of the non-player character can be a reward value of the to-be-performed action calculated based on a reward function after the non-player character performs the to-be-performed action.

[0122] The reward function can determine the reward value of the to-be-performed action based on the next state caused by the to-be-performed action.

[0123] S502: determining an updated Q value corresponding to the to-be-performed action according to the next state of the non-player character, the reward value of the next state, and the Q value of the to-be-performed action in the next state in the current behavior Q value table and the Q value of the to-be-performed action in the current state in the current behavior Q value table.

[0124] After performing the to-be-performed action and determining the next state of the non-player character, the to-be-performed action of the next state can be determined among each action of the next state, and the determination of the to-be-performed action of the next state can be the same as the above S301 step, which will not be repeated here.

[0125] It should be noted that the behavior Q value table in the present application is constantly updated during the training game of the non-player character, so after a plurality of training games, the Q value of each state in the behavior Q value table has a historical value, and the Q value of the to-be-performed action in the next state in the current behavior Q value table can be the historical value of the to-be-performed action of the next state.

[0126] For example, for each time step t, the robot observes the current state S_t, selects a to-be-performed action A_t according to the current behavior Q value table and the ε-greedy strategy, performs the to-be-performed action and determines the next state S_t+1 and the reward R_t+1, and then determines the updated Q value Q'(S_t,A_t) corresponding to the to-be-performed action according to the next state and the current behavior Q value table, that is:

[0127] Q'(S_t,A_t)=Q(S_t,A_t)+α*[R_t+1+γ*Q(S_t+1,A_t+1)-Q(S_t,A_t)]

[0128] Wherein, α is the learning rate, γ is the discount factor, which controls the decay rate of future rewards. Q(S_t+1,A_t+1) represents the Q value of the to-be-performed action in the next state in the current behavior Q value table, R_t+1 represents the reward value of the next state, and Q(S_t,A_t) represents the Q value of the to-be-performed action in the current state in the current behavior Q value table.

[0129] By taking the Q value of the to-be-executed action in the next state as a calculation factor of the Q value of the to-be-executed action, it is ensured that the to-be-executed action in the next state also has an impact on the Q value of the to-be-executed action in the current state, so that the Q value of the to-be-executed action is optimal in combination with the current state and the next state.

[0130] It is worth noting that the above steps S501-S502 are an explanation of determining a to-be-executed action and updating the behavior Q value table, and it should be understood that after updating the behavior Q value table, the next state can be taken as a new current state, and the above steps S501-S502 can be repeatedly executed until the training game ends, and the behavior Q value table updated at the end is taken as the behavior Q value table of the training game.

[0131] The training game includes a non-player character and a human player, and after the training game ends, in order to avoid too large difference between the training result and the demand of the player, the application can also judge the pros and cons of the training game in combination with the feedback information of the player for the training game after the training game ends.

[0132] As shown in Figure 6 The step S304 includes:

[0133] S601: receiving training feedback information of a human player for a current training game.

[0134] After the current training game ends, the electronic device can receive the training feedback information of the human player for the current training game, and the training feedback information can indicate whether the operation of the non-player character in the current training game is reasonable, and the difficulty of the current training game, etc.

[0135] S602: determining whether the updated current behavior Q value table takes effect according to the training feedback information.

[0136] After collecting the training feedback information of the human player for the current training game, the electronic device can analyze the training feedback information to determine whether to discard the result of the current training game.

[0137] S603: if yes, taking the updated current behavior Q value table as the current behavior Q value table of the next training game.

[0138] As a possible implementation, if the training feedback information indicates that the difficulty of the current training game is moderate, and the non-player character does not have unreasonable operations in the current training game, the updated current behavior Q-value table of the current training game can be used as the current behavior Q-value table of the next training game, and in the process of training the next training game, the action to be executed can be selected based on the behavior Q-value table, and the Q-value of the action to be executed can be calculated.

[0139] For example, unreasonable operations of the non-player character can include that the actual participation time is less than the normal game time, and the action executed by the non-player character is against common sense.

[0140] S604: If not, the current behavior Q-value table is used as the current behavior Q-value table of the next training game.

[0141] As another possible implementation, if the training feedback information indicates that the difficulty of the current training game is too simple or too difficult, and the non-player character has unreasonable operations in the current training game, the electronic device can discard the training result of the current training game, that is, the updated behavior Q-value table of the current training game is not used as the current behavior Q-value table of the next training game, but the current behavior Q-value table of the current training game, that is, the updated behavior Q-value table of the last training game of the current training game, is used as the current behavior Q-value table of the next training game.

[0142] It should be noted that, in order to prevent the non-player character from being too strong or too weak to affect the game experience of the player and the game victory and defeat, the win rate of the non-player character in the game can also be limited, and the win rate of the non-player character is constrained within a preset win rate range, for example, if the preset win rate range is 40%-55%, the game data of the non-player character can be dynamically adjusted based on the game performance of the real player, such as critical hit rate or probability of appearance of hidden props, so that the current game is matched with the game level of the player.

[0143] In the embodiments of the present application, by taking the training feedback information of the user for the training game as an influencing factor for training the non-player character, the non-player character obtained by training can be closer to the real player and more meet the real game needs of the player.

[0144] The following is a further description of the above determination of the game operation result of the non-player character in the simulation game and the historical game data of the non-player character to determine the game data change information of the non-player character. It should be noted that before the simulation game, the developer can modify the game parameters first. For example, if the strength of the new character is adjusted after the new character is introduced to ensure the balance of the game, the configuration information of the new character can be modified, such as modifying the damage value, blood volume, and skill refresh frequency of the new character.

[0145] To improve the reliability of the game data, the non-player characters can participate in multiple simulated games, and according to the game operation results of the non-player characters in the simulated games, obtain the game data of the non-player characters in the simulated games, and statistically obtain the simulated game data of the non-player characters in the simulated games.

[0146] The first game data can be statistical information of the game data of the non-player characters in the first simulated game before the game parameters are modified, including the win rate and ranking of the non-player characters.

[0147] The second game data can be statistical information of the game data of the non-player characters in the second simulated game corresponding to the rating after the game parameters are modified, including the win rate and ranking of the non-player characters in the second simulated game.

[0148] The following is a further description of determining the game data change information of the non-player characters according to the first game data and the second game data, as shown in Figure 7 The S204 step includes:

[0149] S701: Determine the win rate difference of the win rate of the first game data of each non-player character and the win rate of the second game data of each non-player character.

[0150] Optionally, the average value of the win rate of the first game data of each non-player character and the average value of the win rate of the second game data can be calculated, and the difference between the average values of the win rates of each non-player character is taken as the win rate difference of the non-player character.

[0151] S702: Determine the ranking difference of the ranking of the first game data of each non-player character and the ranking of the second game data of each non-player character.

[0152] Optionally, the average value of the ranking of the first game data of each non-player character and the average value of the ranking of the second game data can be calculated, and the difference between the average values of the rankings of each non-player character is taken as the ranking difference of the non-player character.

[0153] S703: Take the win rate difference and / or the ranking difference as the game data change information of the non-player character.

[0154] In the present application, the first game data and the second game data of the same rating of the same character can be compared to obtain the win rate difference and the ranking difference of the character in the corresponding rating, and the win rate difference and the ranking difference are taken as the game data change information of the non-player character.

[0155] It is worth noting that the game balance verification can be performed according to the game data change information of all non-player characters in the application.

[0156] It is worth noting that whether the game balance is affected by the parameter modification can also be determined by other indicators in the game data, that is, the game data change information can also include change information of other data, and the application only gives a possible implementation manner by taking the win rate and the ranking as an example, and should not be limited thereto.

[0157] After determining the game data change information, the game state of the asymmetric game can be determined based on the game data change information, and it can be determined whether the game parameters need to be modified. The S205 step includes:

[0158] If the win rate difference is greater than or equal to the preset win rate change threshold, and / or, the ranking difference is greater than or equal to the preset ranking change threshold, it is determined that the asymmetric game is in a non-balanced state, and the second game parameter is modified based on the non-balanced state to obtain a third game parameter.

[0159] If the win rate difference is greater than or equal to the preset win rate change threshold, or the ranking difference is greater than or equal to the preset ranking change threshold, or the win rate difference is greater than or equal to the preset win rate change threshold and the ranking difference is greater than or equal to the preset ranking change threshold, it is determined that the asymmetric game is in a non-balanced state, and the designer can continue to modify the second game parameter to obtain a third game parameter.

[0160] For example, assuming that under the second game parameter, the win rate difference of the game character A in the second game parameter is greater than the win rate change difference, the second game parameter can be adjusted or the parameter of the game character A can be adjusted alone, for example, the single damage value of the character A is reduced.

[0161] As shown in Figure 8 After the third game parameter is obtained by modification, the method of the application further includes:

[0162] S801: Obtain third game data of the trained non-player character in a third simulation game of the asymmetric game, and the third simulation game includes a third game parameter obtained by modifying the second game parameter.

[0163] Under the third game parameter, a third simulation game composed of non-player characters can be created, and third game data of the third simulation game can be obtained.

[0164] S802: Determine game data change information of the non-player character according to the third game data and the second game data.

[0165] S803: Determine the game state of the asymmetric game based on the game data change information, to determine whether to modify the third game parameter based on the game state.

[0166] In the present application, the difference in win rate and the difference in ranking between the second game data and the third game data can be calculated to obtain the game data change information of the third simulated game compared to the second simulated game, and then the game state of the asymmetric game under the third game parameter is determined based on the game data change information.

[0167] If the game state of the asymmetric game is a non-balanced state, the electronic device can output a prompt information to prompt the designer to modify the third game parameter to obtain a fourth game parameter, and perform a fourth simulated game under the fourth game parameter, and then determine the game data change information according to the fourth game data of the fourth simulated game and the third game data, and continue to determine the game state of the asymmetric game based on the game data change information until the game state of the asymmetric game is balanced, and the last modified game parameter is determined as the final game parameter of the asymmetric game.

[0168] As another possible implementation, the step of determining the game state of the asymmetric game in S205 and determining whether to modify the second game parameter based on the game state further comprises:

[0169] If the difference in win rate is less than a preset win rate change threshold, and / or, the difference in ranking is less than a preset ranking change threshold, it is determined that the asymmetric game is in a balanced state, and no modification is made to the second game parameter based on the balanced state.

[0170] If the difference in win rate is less than a preset win rate change threshold, or the difference in ranking is less than a preset ranking change threshold, or both the difference in win rate and the difference in ranking are less than the preset win rate change threshold and the preset ranking change threshold, it can be determined that the parameter modification does not affect the balance of the game, and the asymmetric game is in a balanced state, so no modification is needed for the game parameter.

[0171] When the asymmetric game is in a balanced state, the last modified game parameter can be used as the formal game parameter, and the game can be run when it is online.

[0172] In the embodiments of the present application, the non-player character is first trained through human-computer training games to obtain a non-player character of all ranks and all characters, and after modifying the game parameter, the trained non-player character is used to perform a simulated game, and the game data change information is determined according to the second game data after modifying the parameter and the first game data before modifying the parameter, so as to determine whether the game is in a balanced state based on the game data change information, and determine whether the game parameter needs to be modified, thereby achieving the dual goals of modifying the game parameter without affecting the overall balance of the game, and further optimizing the balance of the game.

[0173] Based on the same inventive concept, the application also provides a data processing device for an asymmetric game corresponding to the data processing method for an asymmetric game. Since the device solves problems in the same principle as the data processing method for an asymmetric game, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0174] Figure 9 The structure of the data processing device for an asymmetric game is shown.

[0175] The training module 901 is configured to iteratively update the behavior Q-value table of the non-player character based on the game result of the non-player character in the training game of the asymmetric game until the trained non-player character is obtained when the iteration termination condition is met. The behavior Q-value table is used to represent the Q-value corresponding to the state-action of the non-player character.

[0176] The first acquisition module 902 is configured to acquire first game data of the trained non-player character in a first simulation game of the asymmetric game. The first simulation game includes the first game parameter before modification.

[0177] The second acquisition module 903 is configured to acquire second game data of the trained non-player character in a second simulation game of the asymmetric game. The second simulation game includes the second game parameter after modification of the first game parameter.

[0178] The change determination module 904 is configured to determine the game data change information of the non-player character according to the second game data and the first game data.

[0179] The state determination module 905 is configured to determine the game state of the asymmetric game based on the game data change information, so as to determine whether to modify the second game parameter based on the game state. The game state includes a balanced state and an unbalanced state.

[0180] In a feasible implementation, the training module 901 is specifically configured to:

[0181] Determine the to-be-executed action of the non-player character according to the current state of the non-player character in the current training game and the action source information. The action source information includes the current behavior Q-value table or the action selection strategy. The current behavior Q-value table is used to represent the Q-value corresponding to the state-action of the non-player character in the current training game.

[0182] Trigger the non-player character to execute the to-be-executed action.

[0183] determine an updated Q value corresponding to the to-be-executed action according to a result of the non-player character executing the to-be-executed action, and update the current behavior Q value table according to the updated Q value corresponding to the to-be-executed action;

[0184] obtain an updated current behavior Q value table at the end of the current training game, and obtain a current behavior Q value table of a next training game according to the updated current behavior Q value table;

[0185] determine whether the current behavior Q value table satisfies an iteration termination condition, and if so, take the updated current behavior Q value table obtained in the last training game as the behavior Q value table of the non-player character.

[0186] In a feasible implementation, the training module 901 is specifically configured to:

[0187] If the Q values in the current behavior Q value table converge, or the number of iterations of the current behavior Q value table is equal to a preset iteration number value, the current behavior Q value table satisfies the iteration termination condition, and the updated current behavior Q value table obtained in the last training game is taken as the behavior Q value table of the non-player character.

[0188] In a feasible implementation, the training module 901 is specifically configured to:

[0189] If the action source information is the current behavior Q value table, the Q values of the actions in the current state are read from the current behavior Q value table.

[0190] According to the Q values of the actions in the current state, the to-be-executed action of the non-player character is determined.

[0191] In a feasible implementation, the training module 901 is specifically configured to:

[0192] If the action source information is the action selection strategy, the to-be-executed action of the non-player character is determined based on the action selection strategy.

[0193] In a feasible implementation, the training module 901 is specifically configured to:

[0194] According to a result of the non-player character executing the to-be-executed action, a next state of the non-player character and a reward value of the next state are determined.

[0195] According to the next state of the non-player character, the reward value of the next state, and the Q value of the to-be-executed action in the next state in the current behavior Q value table and the Q value of the to-be-executed action in the current state in the current behavior Q value table, an updated Q value corresponding to the to-be-executed action is determined.

[0196] In a feasible implementation, the training module 901 is specifically configured to:

[0197] receiving training feedback information of a human player for the current training game;

[0198] determining whether the updated current behavior Q-value table is effective according to the training feedback information;

[0199] if yes, taking the updated current behavior Q-value table as a current behavior Q-value table of a next training game;

[0200] if no, taking the current behavior Q-value table as the current behavior Q-value table of the next training game.

[0201] In an implementable embodiment, the first game data and the second game data each include a win rate of the non-player character and / or a ranking of the non-player character.

[0202] In an implementable embodiment, the change determining module 904 is specifically configured to:

[0203] determine a win rate difference between the win rate of the first game data of each non-player character and the win rate of the second game data of each non-player character;

[0204] determine a ranking difference between the ranking of the first game data of each non-player character and the ranking of the second game data of each non-player character;

[0205] take the win rate difference and / or the ranking difference as the game data change information of the non-player character.

[0206] In an implementable embodiment, the state determining module 905 is specifically configured to:

[0207] if the win rate difference is greater than or equal to a preset win rate change threshold and / or the ranking difference is greater than or equal to a preset ranking change threshold, determine that the asymmetric game is in an unbalanced state, and modify the second game parameter based on the unbalanced state to obtain a third game parameter.

[0208] In an implementable embodiment, the state determining module 905 is specifically configured to:

[0209] obtain third game data of the trained non-player character in a third simulation game of the asymmetric game, the third simulation game including the third game parameter obtained by modifying the second game parameter;

[0210] determine the game data change information of the non-player character according to the third game data and the second game data;

[0211] determine a game state of the asymmetric game based on the game data change information, to determine whether to modify the third game parameter based on the game state.

[0212] In an implementable embodiment, the state determining module 905 is specifically configured to:

[0213] If the win rate difference is less than a preset win rate change threshold, and / or, the ranking difference is less than a preset ranking change threshold, it is determined that the asymmetric game is in a balanced state, and no modification is made to the second game parameter based on the balanced state.

[0214] By training the game robot in the training game and obtaining the behavior Q value table of the game robot according to the game operation result of the game robot in the training game, a game robot that is full of all ranks and all roles and is closer to the level of a real human player is obtained. After the game robot is obtained, the game robot after training is used to perform a simulated game. Not only can different player levels be simulated, but also a large amount of real and objective game data can be obtained in a short time, and problems and defects existing in the game can be quickly found in a short time, so that timely repair and optimization can be performed. After a large amount of game data is obtained by using the game robot to perform a simulated game, the balance of the game can be quickly measured based on the game data and the data before the modification parameter, and the efficiency and effect of the game balance optimization are improved.

[0215] Figure 10 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown, which includes a processor 1001, a storage medium 1002, and a bus 1003. The storage medium 1002 stores machine-readable instructions executable by the processor 1001. When the electronic device runs a data processing method of an asymmetric game as in an embodiment, the processor 1001 and the storage medium 1002 communicate through the bus 1003. The processor 1001 executes the machine-readable instructions, and the processor 1001 performs the pre-sequential part of the method to perform the following steps:

[0216] Based on the game result of the non-player character in the training game of the asymmetric game, iteratively update the behavior Q value table of the non-player character until the iteration termination condition is met to obtain the trained non-player character. The behavior Q value table is used to represent the Q value corresponding to the state action of the non-player character.

[0217] Obtain first game data of the trained non-player character in a first simulated game of the asymmetric game. The first simulated game includes a first game parameter before modification.

[0218] Obtain second game data of the trained non-player character in a second simulated game of the asymmetric game. The second simulated game includes a second game parameter after modification of the first game parameter.

[0219] Determine the game data change information of the non-player character according to the second game data and the first game data.

[0220] Determine a game state of the asymmetric game based on the data change information, and determine whether to modify the second game parameter based on the game state; the game state includes a balanced state and an unbalanced state.

[0221] In an implementation, the processor 1001 is configured to perform the following steps:

[0222] Determine the action to be performed by the non-player character based on the current state of the non-player character in the current training game and the action source information, wherein the action source information includes the current action Q-value table or the action selection strategy, and the current action Q-value table is used to represent the Q-value corresponding to the state-action pair of the non-player character in the current training game.

[0223] Trigger the non-player character to perform the action to be performed.

[0224] Determine the updated Q-value corresponding to the action to be performed based on the result of the non-player character performing the action to be performed, and update the current action Q-value table based on the updated Q-value corresponding to the action to be performed.

[0225] At the end of the current training game, obtain the updated current action Q-value table, and obtain the current action Q-value table of the next training game based on the updated current action Q-value table.

[0226] Determine whether the current action Q-value table meets the iteration termination condition, and if so, use the updated current action Q-value table obtained in the last training game as the action Q-value table of the non-player character.

[0227] In an implementation, the processor 1001 is configured to perform the following steps:

[0228] If the Q-value in the current action Q-value table converges, or the iteration number of the current action Q-value table is equal to the preset iteration number value, the current action Q-value table meets the iteration termination condition, and the updated current action Q-value table obtained in the last training game is used as the action Q-value table of the non-player character.

[0229] In an implementation, the processor 1001 is configured to perform the following steps:

[0230] If the action source information is the current behavior Q-value table, the Q-value of each action in the current state is read from the current behavior Q-value table;

[0231] According to the Q-value of each action in the current state, the action to be performed by the non-player character is determined.

[0232] In a feasible implementation, when the processor 1001 performs the determination of the action to be performed by the non-player character according to the current state of the non-player character in the current training game and the action source information, the processor 1001 is specifically configured to:

[0233] If the action source information is the action selection strategy, the action to be performed by the non-player character is determined based on the action selection strategy.

[0234] In a feasible implementation, when the processor 1001 performs the determination of the updated Q-value corresponding to the action to be performed according to the result of the execution of the action to be performed by the non-player character, the processor 1001 is specifically configured to:

[0235] According to the result of the execution of the action to be performed by the non-player character, the next state of the non-player character and the reward value of the next state are determined;

[0236] According to the next state of the non-player character, the reward value of the next state, the Q-value of the action to be performed in the next state in the current behavior Q-value table, and the Q-value of the action to be performed in the current state in the current behavior Q-value table, the updated Q-value corresponding to the action to be performed is determined.

[0237] In a feasible implementation, when the processor 1001 performs the obtaining of the current behavior Q-value table of the next training game according to the updated current behavior Q-value table, the processor 1001 is specifically configured to:

[0238] Receiving training feedback information of a real human player for the current training game;

[0239] Determining whether the updated current behavior Q-value table takes effect according to the training feedback information;

[0240] If yes, the updated current behavior Q-value table is taken as the current behavior Q-value table of the next training game;

[0241] If no, the current behavior Q-value table is taken as the current behavior Q-value table of the next training game.

[0242] In a feasible implementation, when the processor 1001 performs the determination of the game data change information of the non-player character according to the second game data and the first game data, the processor 1001 is specifically configured to:

[0243] Determining the win rate of the first game data of each non-player character and the win rate difference between the win rate of the first game data of each non-player character and the win rate of the second game data of each non-player character;

[0244] determining a ranking difference between the ranking of the first game data of each non-player character and the ranking of the second game data of each non-player character;

[0245] using the win rate difference and / or the ranking difference as the game data change information of the non-player character.

[0246] In an implementation, the processor 1001 is configured to determine a game state of the asymmetric game based on the game data change information, and determine whether to modify the second game parameter based on the game state.

[0247] If the win rate difference is greater than or equal to a preset win rate change threshold, and / or, the ranking difference is greater than or equal to a preset ranking change threshold, it is determined that the asymmetric game is in an unbalanced state, and the second game parameter is modified based on the unbalanced state to obtain a third game parameter.

[0248] In an implementation, the processor 1001 is further configured to:

[0249] obtain third game data of the trained non-player character in a third simulated game of the asymmetric game, the third simulated game including the third game parameter obtained by modifying the second game parameter;

[0250] determine the game data change information of the non-player character based on the third game data and the second game data;

[0251] determine a game state of the asymmetric game based on the game data change information, and determine whether to modify the third game parameter based on the game state.

[0252] In an implementation, the processor 1001 is configured to determine a game state of the asymmetric game based on the game data change information, and determine whether to modify the second game parameter based on the game state.

[0253] If the win rate difference is less than the preset win rate change threshold, and / or, the ranking difference is less than the preset ranking change threshold, it is determined that the asymmetric game is in a balanced state, and the second game parameter is not modified based on the balanced state.

[0254] The non-player character is trained in the training game, and the behavior Q value table of the non-player character is updated according to the game operation result of the non-player character in the training game, so that the non-player character of all levels and all characters is obtained, which is closer to the level of a real human player. After obtaining the non-player character, the non-player character after training is used to simulate the game, which can not only simulate the game process of different players, but also obtain a large amount of real and objective game data in a short time, and quickly find problems and defects in the game in a short time, so as to facilitate timely repair and optimization. After a large amount of game data is obtained by using the non-player character to simulate the game, the balance of the game can be quickly measured based on the game data and the data before the modification parameters, and the efficiency and effect of the game balance optimization are improved.

[0255] The embodiment of the application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor, and the processor executes the following steps:

[0256] The behavior Q value table of the non-player character is iteratively updated based on the game result of the non-player character in the training game of the asymmetric game, until the trained non-player character is obtained when the iteration termination condition is met; and the behavior Q value table is used to represent the Q value corresponding to the state action of the non-player character.

[0257] The first game data of the trained non-player character in the first simulation game of the asymmetric game is obtained; and the first simulation game includes the first game parameter before modification.

[0258] The second game data of the trained non-player character in the second simulation game of the asymmetric game is obtained, and the second simulation game includes the second game parameter after the first game parameter is modified.

[0259] The game data change information of the non-player character is determined according to the second game data and the first game data.

[0260] Based on the game data change information, the game state of the asymmetric game is determined, so as to determine whether to modify the second game parameter based on the game state; the game state includes a balanced state and an unbalanced state.

[0261] In a feasible implementation, when the processor executes the step of iteratively updating the behavior Q value table of the non-player character based on the game result of the non-player character in the training game of the asymmetric game, until the trained non-player character is obtained when the iteration termination condition is met, the processor is specifically used for:

[0262] determine the to-be-executed action of the non-player character according to the current state of the non-player character in the current training game and action source information, wherein the action source information comprises: a current behavior Q value table or an action selection strategy, and the current behavior Q value table is used to represent Q values corresponding to state-action pairs of the non-player character in the current training game;

[0263] trigger the non-player character to execute the to-be-executed action;

[0264] determine an updated Q value corresponding to the to-be-executed action according to a result of the non-player character executing the to-be-executed action, and update the current behavior Q value table according to the updated Q value corresponding to the to-be-executed action;

[0265] obtain an updated current behavior Q value table at the end of the current training game, and obtain a current behavior Q value table of a next training game according to the updated current behavior Q value table;

[0266] determine whether the current behavior Q value table meets an iteration termination condition, and if so, take the updated current behavior Q value table obtained in the last training game as a behavior Q value table of the non-player character.

[0267] In a feasible implementation, when the processor executes the determination of whether the current behavior Q value table meets the iteration termination condition, and if so, the taking of the updated current behavior Q value table obtained in the last training game as the behavior Q value table of the non-player character, the processor is specifically configured to:

[0268] if the Q values in the current behavior Q value table converge, or the number of iterations of the current behavior Q value table is equal to a preset iteration number value, the current behavior Q value table meets the iteration termination condition, and the updated current behavior Q value table obtained in the last training game is taken as the behavior Q value table of the non-player character.

[0269] In a feasible implementation, when the processor executes the determination of the to-be-executed action of the non-player character according to the current state of the non-player character in the current training game and the action source information, the processor is specifically configured to:

[0270] if the action source information is the current behavior Q value table, read the Q values of the actions in the current state from the current behavior Q value table;

[0271] determine the to-be-executed action of the non-player character according to the Q values of the actions in the current state.

[0272] In a feasible implementation, when the processor executes the determination of the to-be-executed action of the non-player character according to the current state of the non-player character in the current training game and the action source information, the processor is specifically configured to:

[0273] if the action source information is the action selection strategy, determine the to-be-executed action of the non-player character based on the action selection strategy.

[0274] In an implementation, the processor, when determining the updated Q value corresponding to the to-be-executed action according to the result of the non-player character executing the to-be-executed action, is specifically configured to:

[0275] determine a next state of the non-player character and a reward value of the next state according to the result of the non-player character executing the to-be-executed action;

[0276] determine the updated Q value corresponding to the to-be-executed action according to the next state of the non-player character, the reward value of the next state, and a Q value of the to-be-executed action in the next state in the current action Q value table and a Q value of the to-be-executed action in the current state in the current action Q value table.

[0277] In an implementation, the processor, when determining the current action Q value table of the next training game according to the updated current action Q value table, is specifically configured to:

[0278] receive training feedback information of the real human player for the current training game;

[0279] determine whether the updated current action Q value table takes effect according to the training feedback information;

[0280] if yes, take the updated current action Q value table as the current action Q value table of the next training game;

[0281] if no, take the current action Q value table as the current action Q value table of the next training game.

[0282] In an implementation, the processor, when determining the game data change information of the non-player character according to the second game data and the first game data, is specifically configured to:

[0283] determine a win rate difference between a win rate of the first game data of each non-player character and a win rate of the second game data of each non-player character;

[0284] determine a ranking difference between a ranking of the first game data of each non-player character and a ranking of the second game data of each non-player character;

[0285] take the win rate difference and / or the ranking difference as the game data change information of the non-player character.

[0286] In an implementation, the processor, when determining the game state of the asymmetric game based on the game data change information, and determining whether to modify the second game parameter based on the game state, is specifically configured to:

[0287] If the win rate difference is greater than or equal to a preset win rate change threshold, and / or, the ranking difference is greater than or equal to a preset ranking change threshold, it is determined that the asymmetric game is in an unbalanced state, and the second game parameter is modified based on the unbalanced state to obtain a third game parameter.

[0288] In a feasible implementation, the processor is further configured to:

[0289] Obtain third game data of the trained non-player character in a third simulated game of the asymmetric game, the third simulated game including the third game parameter obtained by modifying the second game parameter;

[0290] Determine game data change information of the non-player character according to the third game data and the second game data;

[0291] Based on the game data change information, determine a game state of the asymmetric game, to determine whether to modify the third game parameter based on the game state.

[0292] In a feasible implementation, when the processor determines the game state of the asymmetric game based on the game data change information, to determine whether to modify the second game parameter based on the game state, the processor is specifically configured to:

[0293] If the win rate difference is less than the preset win rate change threshold, and / or, the ranking difference is less than the preset ranking change threshold, it is determined that the asymmetric game is in a balanced state, and the second game parameter is not modified based on the balanced state.

[0294] By training the non-player character in the training game and obtaining the behavior Q value table of the non-player character according to the game operation result of the non-player character in the training game, a non-player character that is full of all ranks and all characters, and is closer to the level of a real human player, is obtained. After the non-player character is obtained, the trained non-player character is used to perform a simulated game. Not only can the game process of different player levels be simulated, but also a large amount of real and objective game data can be obtained in a short time, and problems and defects existing in the game can be quickly found in a short time, so that timely repair and optimization can be performed. After a large amount of game data is obtained by using the non-player character to perform a simulated game, the balance of the game can be quickly measured based on the game data and the data before the modification parameter, and the efficiency and effect of the game balance optimization are improved.

[0295] In the embodiments of the present application, the computer program can also execute other machine-readable instructions when executed by the processor to perform the methods described in the embodiments. For specific method steps and principles, refer to the description of the embodiments, which will not be described in detail here.

[0296] In the embodiments of the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. The embodiments described above are merely exemplary, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, and electrical, mechanical or other forms.

[0297] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0298] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0299] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0300] It should be noted that: similar reference numerals and letters in the following drawings represent similar items, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0301] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present application, and are used to illustrate the technical solutions of the present application, but not to limit the same. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any person skilled in the art can make modifications or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features within the technical scope disclosed by the present application. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. All should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data processing method of an asymmetric game, characterized by, The method comprises: iteratively updating a behavior Q-value table of the non-player character based on game session results of the non-player character in training sessions of the asymmetric game until a trained non-player character is obtained when an iteration termination condition is met, the behavior Q-value table being used to represent Q-values corresponding to state actions of the non-player character; obtaining first session data of the trained non-player character in a first simulated session of the asymmetric game; the first simulated session comprising first game parameters before modification; obtaining second session data of the trained non-player character in a second simulated session of the asymmetric game, the second simulated session comprising second game parameters after modification of the first game parameters; determining session data change information of the non-player character based on the second session data and the first session data; based on the session data change information, determining a game state of the asymmetric game to determine whether to modify the second game parameters based on the game state; the game state comprising a balanced state and an unbalanced state, the balanced state being determined based on a win rate difference and / or a ranking difference of the non-player character; the method of iteratively updating a behavior Q-value table of the non-player character based on game session results of the non-player character in training sessions of the asymmetric game until a trained non-player character is obtained when an iteration termination condition is met, comprises: determining a to-be-executed action of the non-player character based on a current state of the non-player character in a current training session and action source information, wherein the action source information comprises a current behavior Q-value table or an action selection strategy, the current behavior Q-value table being used to represent Q-values corresponding to state actions of the non-player character in the current training session; triggering the non-player character to execute the to-be-executed action; determining an updated Q-value corresponding to the to-be-executed action based on a result of the non-player character executing the to-be-executed action, and updating the current behavior Q-value table based on the updated Q-value corresponding to the to-be-executed action; obtaining an updated current behavior Q-value table at the end of the current training session, and obtaining a next training session current behavior Q-value table based on the updated current behavior Q-value table; determining whether the current behavior Q-value table meets an iteration termination condition, and if so, taking the updated current behavior Q-value table obtained in the last training session as the behavior Q-value table of the non-player character; wherein if the Q-value in the current behavior Q-value table meets a preset convergence condition, or the number of iteration updates is equal to a preset number of iterations, it is determined that the current behavior Q-value table meets the iteration termination condition.

2. The method of claim 1, wherein, The method of determining a to-be-executed action of the non-player character based on a current state of the non-player character in a current training session and action source information, comprises: if the action source information is the current behavior Q-value table, reading Q-values of actions in the current state from the current behavior Q-value table; determining the to-be-executed action of the non-player character based on the Q-values of the actions in the current state.

3. The method of claim 1, wherein, The determining the action to be executed by the non-player character according to the current state of the current training game and the action source information comprises: If the action source information is the action selection strategy, determining the action to be executed by the non-player character based on the action selection strategy.

4. The method of claim 1, wherein, The determining the updated Q value corresponding to the action to be executed according to the result of the non-player character executing the action to be executed comprises: Determining the next state of the non-player character and the reward value of the next state according to the result of the non-player character executing the action to be executed; Determining the updated Q value corresponding to the action to be executed according to the next state of the non-player character, the reward value of the next state, the Q value of the action to be executed in the next state in the current action Q value table and the Q value of the action to be executed in the current state in the current action Q value table.

5. The method of claim 1, wherein, The obtaining the current action Q value table of the next training game according to the updated current action Q value table comprises: Receiving the training feedback information of the real human player for the current training game; Determining whether the updated current action Q value table takes effect according to the training feedback information; If yes, taking the updated current action Q value table as the current action Q value table of the next training game; If no, taking the current action Q value table as the current action Q value table of the next training game.

6. The method according to any one of claims 1 to 5, characterized in that, The first game data and the second game data both comprise the win rate of the non-player character and / or the ranking of the non-player character.

7. The method of claim 6, wherein, The determining the game data change information of the non-player character according to the second game data and the first game data comprises: Determining the win rate difference between the win rate of the first game data of each non-player character and the win rate of the second game data of each non-player character; Determining the ranking difference between the ranking of the first game data of each non-player character and the ranking of the second game data of each non-player character; Taking the win rate difference and / or the ranking difference as the game data change information of the non-player character.

8. The method of claim 7, wherein, The determining the game state of the asymmetric game based on the game data change information, to determine whether to modify the second game parameter based on the game state, comprises: If the win rate difference is greater than or equal to a preset win rate change threshold and / or the ranking difference is greater than or equal to a preset ranking change threshold, determining that the asymmetric game is in the unbalanced state, and modifying the second game parameter to obtain a third game parameter based on the unbalanced state. The method further comprises: Obtaining third game data of the non-player character in a third simulated game of the asymmetric game after training, the third simulated game comprising a third game parameter obtained by modifying the second game parameter; Determining the game data change information of the non-player character according to the third game data and the second game data; Determining the game state of the asymmetric game based on the game data change information, to determine whether to modify the third game parameter based on the game state.

9. The method of claim 7, wherein, The game state of the asymmetric game is determined based on the game data change information, and whether the second game parameter is modified is determined based on the game state. If the win rate difference is less than a preset win rate change threshold and / or the ranking difference is less than a preset ranking change threshold, it is determined that the asymmetric game is in the balanced state, and the second game parameter is not modified based on the balanced state.

10. A data processing apparatus for an asymmetric game, characterized by, The method comprises the following steps: The training module is configured to iteratively update a behavior Q value table of the non-player character based on game results of the non-player character in training games of the asymmetric game, until a trained non-player character is obtained when an iteration termination condition is met; the behavior Q value table is used to represent Q values corresponding to state actions of the non-player character. The first obtaining module is configured to obtain first game data of the trained non-player character in a first simulated game of the asymmetric game; the first simulated game comprises a first game parameter before modification. The second obtaining module is configured to obtain second game data of the trained non-player character in a second simulated game of the asymmetric game; the second simulated game comprises a second game parameter after modification of the first game parameter. The change determining module is configured to determine game data change information of the non-player character according to the second game data and the first game data. The state determining module is configured to determine a game state of the asymmetric game based on the game data change information, and to determine whether the second game parameter is modified based on the game state. The game state comprises a balanced state and a non-balanced state, and the balanced state is determined based on a win rate difference and / or a ranking difference of the non-player character. The training module is configured to: determine a to-be-executed action of the non-player character according to a current state of the non-player character in a current training game and action source information, wherein the action source information comprises a current behavior Q value table or an action selection strategy, and the current behavior Q value table is used to represent Q values corresponding to state actions of the non-player character in the current training game; trigger the non-player character to execute the to-be-executed action; determine an updated Q value corresponding to the to-be-executed action according to a result of the non-player character executing the to-be-executed action, and update the current behavior Q value table according to the updated Q value corresponding to the to-be-executed action; obtain an updated current behavior Q value table when the current training game ends, and obtain a current behavior Q value table of a next training game according to the updated current behavior Q value table; determine whether the current behavior Q value table meets an iteration termination condition, and if so, take an updated current behavior Q value table obtained in a last training game as a behavior Q value table of the non-player character; wherein if the Q values in the current behavior Q value table meet a preset convergence condition or the number of iteration updates is equal to a preset number of iterations, it is determined that the current behavior Q value table meets the iteration termination condition.

11. An electronic device, comprising: The method comprises the following steps: A processor, a storage medium storing machine readable instructions executable by the processor, and a bus for communication between the processor and the storage medium when the electronic device is running, the processor executing the machine readable instructions to perform the steps of the data processing method of the asymmetric game of any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, A computer readable storage medium storing a computer program, the computer program being executed by a processor to perform the steps of the data processing method of the asymmetric game of any one of claims 1 to 9.

Images