A game battle model training method, game battle method and related device

Abstract

The embodiments of the present application provide a game battle model training method, a game battle method and related devices, which can conduct game battles with target game battle models selected from a game battle model set, thereby generating battle status information, and training the game battle model based on the battle status information, so that the trained game battle model can be applied in the game battle. In this way, the trained game battle model can be used to predict the position of the enemy team, so that the trained game battle model can observe a game situation that is close to the global state of the game, thereby enhancing the accuracy of the trained game battle model in making game decisions and avoiding decision deviations caused by local game states.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a game battle model training method, a game battle method and related devices, wherein the related devices include a game battle model training device, a game battle device, a computer device and a computer-readable storage medium. Background Art

[0002] In existing multiplayer online battle arena (MOBA) games, players are divided into two hostile camps, and they compete against each other to eventually destroy the enemy's crystal to win the game. Due to the complex terrain environment in multiplayer online battle arena games, such as the presence of grass in the terrain environment, and the limited field of view of the game, the game battle model can only observe the local game state of the enemy camp, but cannot observe the global game state of the enemy camp. As a result, the game battle model cannot make reasonable decisions in the game, resulting in low accuracy of game decisions. Summary of the invention

[0003] The embodiment of the present application provides a game battle model training method, which can improve the accuracy of game decisions.

[0004] A game battle model training method, comprising:

[0005] Obtaining a game battle model set, the game battle model set including a plurality of game battle models, the game battle model including an updated version of the game battle model obtained by iteratively training the initial game battle model;

[0006] Selecting at least two target game battle models from the game battle model set for game battle to generate battle status information of the current team and the enemy team, wherein the target game battle models include the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set;

[0007] Extract the battle status information to obtain battle status features and position labels for the enemy team;

[0008] The latest version of the game battle model is trained based on the battle state features and position labels to obtain a trained game battle model.

[0009] Accordingly, the embodiment of the present application provides a game battle model training device, comprising:

[0010] A first acquisition unit may be used to acquire a game battle model set, the game battle model set including a plurality of game battle models, the game battle model including an updated version of the game battle model obtained by iteratively training the initial game battle model;

[0011] A screening unit may be used to screen out at least two target game battle models from the game battle model set for game battle, so as to generate battle status information between the current team and the enemy team, wherein the target game battle models include the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set;

[0012] The first extraction unit can be used to extract the battle status information to obtain battle status features and a position tag for the enemy team;

[0013] The training unit can be used to train the latest version of the game battle model based on battle state features and position labels to obtain a trained game battle model.

[0014] In some embodiments, the latest version of the game battle model includes an enemy position prediction sub-model and an action decision prediction sub-model; the training unit can be specifically used to train the enemy position prediction sub-model based on battle status features and position labels to obtain a trained enemy position prediction sub-model; based on battle status features, the action decision prediction sub-model is trained to obtain a trained action decision prediction sub-model.

[0015] In some embodiments, the non-latest version of the game battle model includes a non-latest state value sub-model; a training unit, which can be specifically used to perform action prediction based on battle state characteristics using an action decision prediction sub-model to obtain the predicted action probability distribution of the current team for the battle state information; based on the battle state characteristics, a non-latest state value sub-model is used to perform value prediction to obtain the state value for the battle state information; based on the state value and the predicted action probability distribution, the action decision prediction sub-model is trained to obtain a trained action decision prediction sub-model.

[0016] In some embodiments, the training unit can be specifically used to predict the position of the enemy team based on the battle status characteristics by using the enemy position prediction sub-model to obtain the predicted position of the enemy team; based on the battle status characteristics and the battle role characteristics, an action decision prediction sub-model is used to predict the action to obtain the predicted action probability distribution of the current team for the battle status information.

[0017] In some embodiments, the screening unit can be specifically used to obtain a preset probability value; when the preset probability value is greater than or equal to a preset threshold, the latest version of the game battle model and the non-latest version of the game battle model are screened out from the game battle model set for game battle to generate battle status information.

[0018] The present application also provides a game battle method, including:

[0019] Get the battle status information between the current team and the enemy team in the current game;

[0020] Extract features of the battle status information to obtain the battle status features of the current game;

[0021] Based on the battle status characteristics, the trained game battle model is used to predict the position of the enemy team to determine the predicted position of the enemy team, and the trained game battle model is the aforementioned trained game battle model;

[0022] Based on the predicted position of the enemy team, the trained game battle model is used to predict the actions of the current team and control the current team to execute the predicted actions.

[0023] Accordingly, the embodiment of the present application provides a game battle model training device, comprising:

[0024] The second acquisition unit can be used to obtain the battle status information between the current team and the enemy team in the current game;

[0025] The second extraction unit can be used to extract features from the battle status information to obtain the battle status features of the current game;

[0026] A determination unit may be used to predict the position of the enemy team based on the battle state characteristics by using a trained game battle model to determine the predicted position of the enemy team, wherein the trained game battle model is the aforementioned trained game battle model;

[0027] The prediction unit can be used to predict the actions of the current team based on the predicted position of the enemy team, using the trained game battle model, and controlling the current team to perform the predicted actions.

[0028] In some embodiments, the trained game battle model includes a trained enemy position prediction sub-model and a trained action decision prediction sub-model; the determination unit can be specifically used to predict the position of the enemy team based on the battle status characteristics using the trained enemy position prediction sub-model to determine the predicted position of the enemy team; the prediction unit can be specifically used to predict the action of the current team based on the predicted position of the enemy team using the trained action decision prediction sub-model.

[0029] In addition, an embodiment of the present application also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any game battle model training method provided in the embodiment of the present application.

[0030] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for loading by a processor to execute any game battle model training method provided in the embodiment of the present application.

[0031] In addition, an embodiment of the present application also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any one of the game battle methods provided in the embodiment of the present application.

[0032] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for loading by a processor to execute any game battle method provided in the embodiment of the present application.

[0033] The embodiment of the present application can obtain a game battle model set, the game battle model set includes several game battle models, the game battle model includes an updated version of the game battle model obtained by iteratively training the initial game battle model; at least two target game battle models are selected from the game battle model set to perform game battles to generate battle status information between the current team and the enemy team, the target game battle models include the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set; the battle status information is extracted to obtain battle status features and position labels for the enemy team; the latest version of the game battle model is trained based on the battle status features and the position labels to obtain a trained game battle model. Since the embodiments of the present application can conduct a game battle with a target game battle model selected from a game battle model set, thereby generating battle status information, and training the game battle model based on the battle status information, the trained game battle model can be applied in the game battle. In this way, the trained game battle model can be used to predict the position of the enemy team, so that the trained game battle model can observe a game situation that is close to the global state of the game, thereby enhancing the accuracy of the trained game battle model in making game decisions and avoiding decision-making deviations caused by local game states. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 It is a scene schematic diagram of the game battle model training method provided in an embodiment of the present application;

[0036] Figure 2 1 is a flowchart of a method for training a game battle model provided in an embodiment of the present application;

[0037] Figure 3 It is a schematic diagram of extracting features from battle situation information provided in an embodiment of the present application;

[0038] Figure 4 It is a schematic diagram of regional segmentation of a global map provided in an embodiment of the present application;

[0039] Figure 5 The latest version of the game battle model is trained to obtain a schematic diagram of the trained game battle model provided in an embodiment of the present application;

[0040] Figure 6 A schematic diagram of training the action decision prediction sub-model based on the battle situation characteristics and the battle role characteristics provided in the embodiment of the present application to obtain the trained action decision prediction sub-model;

[0041] Figure 7 2 is a flowchart of a method for training a game battle model provided in an embodiment of the present application;

[0042] Figure 8 It is a flowchart of a game battle method provided in an embodiment of the present application;

[0043] Fig. 9 It is a structural schematic diagram of a game battle model training device provided in an embodiment of the present application;

[0044] Fig.10 It is a structural diagram of the game battle method provided in an embodiment of the present application;

[0045] Fig.11 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0047] The embodiment of the present application provides a method, device, computer equipment and computer-readable storage medium for training a game battle model. The game battle model training device can be integrated in a computer equipment, which can be a server or a terminal.

[0048] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0049] The embodiments of the present application relate to artificial intelligence (AI), which is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0050] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0051] For example, see Figure 1 Taking the game battle model training device integrated in a computer device as an example, the computer device obtains a game battle model set; selects at least two target game battle models from the game battle model set to perform game battles to generate battle status information between the current team and the enemy team; extracts the battle status information to obtain battle status features and position labels for the enemy team; trains the latest version of the game battle model based on the battle status features and the position labels to obtain a trained game battle model.

[0052] The game battle model set includes several game battle models, and the game battle models include updated version game battle models obtained by iteratively training the initial game battle models. The initial game battle models refer to game battle models that have not been iteratively trained.

[0053] Among them, the target game battle model includes the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set.

[0054] Among them, the battle status information includes battle situation information and battle role information.

[0055] Battle situation information refers to the battle situation information between the current team and the enemy team, including the global map information of the game, the real-time progress information of the game battle, and the advantages and disadvantages information of the battle between the current team and the enemy team. The real-time progress information of the game battle includes the battle situation between the current team and the enemy team that can be observed by the current team in the game interface. The advantages and disadvantages information includes the difference in head count, economic difference, and tower number between the current team and the enemy team.

[0056] The battle character information includes the player characters, non-player characters, and soldier lines that the current team can observe in the game interface. Non-player characters include wild monsters and goblins.

[0057] The battle character features include the player character features, non-player character features and soldier line features. The player features include the player character's health, the player character's mana, and the player character's skill status.

[0058] Wherein, both the current team and the enemy team include at least one player character.

[0059] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0060] This embodiment will be described from the perspective of a game battle model training device, which can be specifically integrated into a computer device, which can be a server or a terminal. Among them, the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device or other smart devices that can acquire data.

[0061] like Figure 2 As shown, the specific process of the game battle model training method is as follows:

[0062] S101. Obtain a game battle model set.

[0063] The game battle model set includes several game battle models. The game battle models may include an initial game battle model and an updated version of the game battle model obtained by iteratively training the initial game battle model. The initial game battle model refers to a game battle model that has not been iteratively trained.

[0064] The embodiment of the present application iteratively trains the initial game battle model, and stores the initial game battle model and the updated version of the game battle model obtained from each training into a game battle model set. The game battle model set can be stored in the cloud or in a database of a computer device.

[0065] The game battle model set may be provided with a set identifier, so that the computer device may extract the game battle set from a database of the cloud or the computer device based on the set identifier.

[0066] S102: Filter out at least two target game battle models from the game battle model set to perform game battle, so as to generate battle status information between the current team and the enemy team.

[0067] Since in the embodiment of the present application, the teams in the game are controlled by the game battle model, the game battle model can be used to control at least one team to fight. In the embodiment of the present application, the game battle model needs to be trained. Since it is necessary to ensure that the trained game battle model has robustness, based on this, in the historical game battle model of the embodiment of the present application, the target game battle model is selected from the game battle model set to play the game battle and generate battle status information. In this way, the latest version of the game battle model is trained based on the battle status information generated by the historical game battle model battle, so that the robustness of the trained game battle model can be enhanced.

[0068] Based on the above, the target game battle model of the present application includes the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set, so that the latest version of the game battle model can learn the control method of the non-latest version of the game battle model against the battle team, thereby enhancing the robustness of the trained game battle model.

[0069] Among them, the battle status information includes battle situation information and battle role information.

[0070] Battle situation information refers to the battle situation information between the current team and the enemy team, including the global map information of the game, the real-time progress information of the game battle, and the advantages and disadvantages information of the battle between the current team and the enemy team. The real-time progress information of the game battle includes the battle situation between the current team and the enemy team that can be observed by the current team in the game interface. The advantages and disadvantages information includes the difference in head count, economic difference, and tower number between the current team and the enemy team.

[0071] The battle character information includes the player characters, non-player characters, and soldier lines that the current team can observe in the game interface. Non-player characters include wild monsters and goblins.

[0072] Wherein, both the current team and the enemy team include at least one player character.

[0073] In the embodiment of the present application, there are multiple ways to select at least two target game battle models from the game battle model set for game battle, as detailed below:

[0074] For example, in an embodiment of the present application, each game battle model has a model identifier, based on which the computer device can filter out a target game battle model from a set of game battle models based on the model identifier for game battle.

[0075] Among them, the computer device can randomly select a target game battle model from the game battle model set based on the model identifier to perform game battle.

[0076] Among them, in an embodiment of the present application, the target game battle model may include at least one non-latest version game battle model and at least one latest version game battle model.

[0077] Based on the above, for example, when there are two target game battle models, the target game battle model includes the latest version game battle model and the non-latest version game battle model. When there are three target game battle models, the target game battle model may include two latest version game battle models and one non-latest version game battle model, and the target game battle model may include one latest version game battle model and two non-latest version game battle models. When the target game battle model includes multiple, and the non-latest version game battle model includes at least two, the non-latest version game battle model may adopt an updated version game battle model of the same updated version, or may adopt an updated version game battle model of different updated versions.

[0078] For another example, an embodiment of the present application can filter out a target game battle model from a game battle model set based on a preset probability value. Specifically, a computer device obtains a preset probability value; based on the preset probability value, at least two target game battle models are filtered out from the game battle model set for game battle to generate battle status information.

[0079] The preset probability value may be 0.5, but is not limited to 0.5. The preset probability value may be set according to specific requirements, and the preset probability value may also be a value randomly given by a computer device.

[0080] In order to improve the robustness of the game battle model after training, the embodiment of the present application can set a preset threshold value for the preset probability value for judgment, as follows:

[0081] For example, when the preset probability value is less than a preset threshold, at least two target game battle models with the same update version are selected from the game battle model set for game battle to generate battle status information; when the preset probability value is greater than or equal to the preset threshold, the latest version game battle model and the non-latest version game battle model are selected from the game battle model set for game battle to generate battle status information.

[0082] The preset threshold may be 0.9, but is not limited to 0.9, and the preset threshold may be set according to specific requirements.

[0083] In an embodiment of the present application, the computer device can randomly give a preset probability value, and based on the preset probability value, screen out the latest version of the game battle model and the non-latest version of the game battle model for game battle. In this way, the latest version of the game battle model can learn the playing methods of the historical game battle model during the training process, thereby enhancing the robustness of the trained game battle model.

[0084] In an embodiment of the present application, the target game battle model for updating the version of the game battle model can be selected from the latest version of the game battle model in the updated version of the game battle model.

[0085] S103: Extract the battle status information to obtain battle status features and a position label for the enemy team.

[0086] The battle status information includes battle situation information and battle role information, and the battle status features include battle situation features and battle role features.

[0087] The embodiments of the present application can use the latest version of the game battle model to extract features from the battle situation information to obtain battle situation features; use the latest version of the game battle model to extract features from the battle role information to obtain battle role features; use the latest version of the game battle model to extract labels from the battle status information to obtain position labels for the enemy team.

[0088] Of course, the embodiments of the present application may also use other neural network models to extract battle situation information and battle role information to obtain battle situation features, battle role features and position labels for the enemy team.

[0089] Among them, Figure 3As shown, the present application implements feature extraction of global map information of battle situation information to obtain global map features; feature extraction of real-time situation progress information of game battles in the battle situation information to obtain real-time situation progress features; feature extraction of advantage and disadvantage information in the battle situation information to obtain advantage and disadvantage features, wherein the advantage and disadvantage information refers to the advantage and disadvantage information of the battle between the current team and the enemy team.

[0090] Among them, since the global map information and the real-time situation progress information of the game battle are image information, the embodiment of the present application uses a graphical method to extract features from the global map information and the real-time situation progress information of the game battle; since the advantage and disadvantage information includes the head difference, economic difference, and tower number difference between the current team and the enemy team, that is, the advantage and disadvantage information is data information, the embodiment of the present application uses a vectorized method to extract features from the advantage and disadvantage information.

[0091] In the embodiment of the present application, before extracting the position tags of the enemy team, the computer device may pre-segment the global map of the game, for example, Figure 4 As shown, the computer device divides the global map into regions according to the preset distance to determine several regions of the global map; the computer device assigns different numbers to each region. In this way, each region on the global map has a number, so that the location tag of the enemy team can be determined. For example, the enemy team has three characters, character A, character B, and character C. Character A is located in the region numbered 11 on the global map, character B is located in the region numbered 20 on the global map, and character C is located in the region numbered 22 on the global map.

[0092] Based on this, the computer device determines the role of the enemy team based on the characteristics of the battle role; based on the global map characteristics in the battle situation characteristics and the role of the enemy team, the target area on the global map where the character of the enemy team is located is determined; based on the number of the target area, the location tag of the enemy team is determined, that is, the embodiment of the present application can use the number of the target area as the location tag of the enemy team. For example, the embodiment of the present application uses the number 11 of the area in the global map where character A is located, the number 20 of the area in the global map where character B is located, and the number 22 of the area in the global map where character C is located as the location tag of the enemy team.

[0093] S104: Train the latest version of the game battle model based on the battle state features and the position labels to obtain a trained game battle model.

[0094] The game battle model of the embodiment of the present application can be a separate model. The trained game battle model can predict the position of the enemy team and obtain the predicted position of the enemy team. The trained game battle model can also predict the action of the current team and obtain the predicted action of the current team, thereby controlling the current team to perform the predicted action.

[0095] The game battle model of the embodiment of the present application can be a model composed of multiple sub-models. The following will be explained using the latest version of the game battle model as an example.

[0096] The latest version of the game battle model includes an enemy position prediction sub-model and an action decision prediction sub-model. Figure 5 As shown, the computer device trains the latest version of the game battle model to obtain the trained game battle model as follows:

[0097] A1. Based on the battle status features and position labels, the enemy position prediction sub-model is trained to obtain a trained enemy position prediction sub-model.

[0098] Among them, the enemy position prediction sub-model can be a CNN neural network model, or a RNN neural network model. Of course, the enemy position prediction sub-model can also be other types of neural network models.

[0099] Since the battle status characteristics include battle situation characteristics and battle role characteristics, based on this, the computer device can input the battle situation characteristics, battle role characteristics and position labels into the enemy position prediction sub-model; based on the battle situation characteristics and battle role characteristics, the enemy position prediction sub-model is used to predict the position of the enemy team to obtain the predicted position of the enemy team.

[0100] Among them, at time t, the enemy position prediction sub-model predicts the enemy team's predicted position at time t+T, where T is the prediction time interval. For example, at the 3rd second, the enemy position prediction sub-model predicts the enemy team's predicted position at the 5th second.

[0101] For example, the embodiment of the present application divides the global map in the global map information into 12×12 areas, each area has a different number. Based on this, the embodiment of the present application extracts features from the global map information in the battle situation information to obtain global map features, and the global map features are a 12×12 matrix. Based on the global map features and the battle role features, the enemy position prediction sub-model is used to predict the position of the enemy team. The enemy position prediction sub-model is normalized by the softmax function to obtain the probability that a role in the enemy team exists in each area of ​​the global map.

[0102] The embodiment of the present application can filter out the area with the highest probability of the existence of characters in the enemy team as the predicted position of the enemy team. In this way, the embodiment of the present application calculates the loss between the predicted position of the enemy team and the label position; based on the loss between the predicted position of the enemy team and the label position, the enemy position prediction sub-model is trained to obtain the trained enemy position prediction sub-model.

[0103] Among them, the embodiment of the present application can use the cross entropy loss function to calculate the loss between the enemy team's predicted position and the label position, and can also use the logarithmic loss function to calculate the loss between the enemy team's predicted position and the label position. Of course, the embodiment of the present application can use other loss functions to calculate the loss between the enemy team's predicted position and the label position.

[0104] A2. Based on the battle state characteristics, the action decision prediction sub-model is trained to obtain a trained action decision prediction sub-model.

[0105] Among them, the action decision prediction sub-model can be a CNN neural network model, or a RNN neural network model. Of course, the action decision prediction sub-model can also be other types of neural network models.

[0106] The computer device can input the battle state characteristics, that is, the battle situation characteristics and the battle role characteristics, into the action decision prediction sub-model; based on the battle situation characteristics and the battle role characteristics, the action decision prediction sub-model is used to perform action prediction to obtain the predicted action of the current team.

[0107] Among them, the embodiment of the present application can obtain the actual action of the current team; calculate the loss between the actual action and the predicted action; based on the loss between the actual action and the predicted action, train the action decision prediction sub-model to obtain the trained action decision prediction sub-model.

[0108] In addition to the above, the embodiments of the present application can input the battle status characteristics into the action decision prediction sub-model; based on the battle status characteristics, the action decision prediction sub-model is used to predict the position of the enemy team to obtain the predicted position of the enemy team; based on the predicted position of the enemy team and the battle status characteristics, the action decision prediction sub-model is used to predict the action to obtain the predicted action of the current team.

[0109] Similarly, the embodiment of the present application can obtain the actual action of the current team; calculate the loss between the actual action and the predicted action; based on the loss between the actual action and the predicted action, train the action decision prediction sub-model to obtain the trained action decision prediction sub-model.

[0110] In an embodiment of the present application, the latest version of the game battle model also includes a state value sub-model, that is, the latest version of the game battle model includes an enemy position prediction sub-model, an action decision prediction sub-model and a state value sub-model, and accordingly, the non-latest version of the game battle model includes a non-latest enemy position prediction sub-model, a non-latest action decision prediction sub-model and a non-latest state value sub-model.

[0111] Based on this, Figure 6As shown, in the embodiment of the present application, the action decision prediction sub-model is trained based on the battle state characteristics, and the process of obtaining the trained action decision prediction sub-model can be as follows:

[0112] First of all, it should be noted that in the embodiment of the present application, the Actor-Critic algorithm is used to train the action decision prediction sub-model. The architecture of the Actor-Critic algorithm includes two neural network models, namely the Actor network model and the Critic network model. The Actor network model is used to generate strategies, and the Critic network model is used to evaluate the value of the strategies generated by the Actor network model.

[0113] In the embodiment of the present application, the action decision prediction sub-model is the Actor network model, and the state value sub-model is the Critic network model.

[0114] Similarly, the non-latest action decision prediction sub-model of the non-latest version of the game battle model is the Actor network model, and the non-latest state value sub-model in the non-latest version of the game battle model is the Critic network model.

[0115] B1. Based on the battle status characteristics, the action decision prediction sub-model is used to predict actions and obtain the predicted action probability distribution of the current team for the battle status information.

[0116] Specifically, the embodiments of the present application can use an enemy position prediction sub-model to predict the position of the enemy team based on the battle status characteristics to obtain the predicted position of the enemy team; based on the predicted position of the enemy team and the battle status characteristics, an action decision prediction sub-model can be used to predict actions to obtain the predicted action probability distribution of the current team for the battle status information.

[0117] That is, the action decision prediction sub-model can predict actions based on the predicted position of the enemy team and the characteristics of the battle status. Based on this, the trained action decision prediction sub-model obtained after the training of the action decision prediction sub-model can refer to the predicted position of the enemy team and predict the predicted action of the current team, thereby enhancing the accuracy of game decisions.

[0118] The following is further explained. Figure 7 As shown, in the embodiment of the present application, the target game battle model generates battle status information of the current team and the enemy team during the game battle. The embodiment of the present application extracts the battle status information to obtain battle status features and position tags for the enemy team.

[0119] The non-latest action decision prediction sub-model in the non-latest version of the game battle model predicts the initial predicted action probability distribution based on the battle state characteristics of the battle state information. Of course, the non-latest action decision prediction sub-model here predicts the initial predicted action probability distribution based on the battle situation characteristics of the battle state characteristics and the battle role characteristics of the battle state characteristics, as well as the enemy team predicted position predicted by the enemy team prediction sub-model.

[0120] The computer device can select the initial predicted action with the highest probability based on the probability distribution of the initial predicted action, and then execute the initial predicted action, thereby advancing the game process, thereby obtaining new battle state information; extracting features from the new battle state information to obtain new battle state features. Based on this, the non-latest action decision prediction sub-model calculates the reward score for the initial predicted action. The new battle state features include new battle situation features and new battle role features.

[0121] B2. Based on the battle status characteristics, a non-latest state value sub-model is used to perform value prediction to obtain the state value for the battle status information.

[0122] The state value includes the initial state value and the candidate state value. The state value represents the influence of the battle state information generated during the process on the predicted action. The state value can be expressed in a score.

[0123] Specifically, during the game battle process of the non-latest version of the game battle model, the non-latest state value sub-model in the non-latest version of the game battle model calculates the initial state value based on the battle state characteristics, and the initial state value represents the value of the battle situation characteristics and the battle character characteristics to the action prediction.

[0124] The non-latest state value sub-model calculates the candidate state value based on the new battle state features, and the candidate state value represents the value of the new battle state features for action prediction; and generates the state value based on the initial state value and the candidate state value.

[0125] B3. Based on the state value and the predicted action probability distribution, the action decision prediction sub-model is trained to obtain the trained action decision prediction sub-model.

[0126] Among them, based on the initial state value and the candidate state value in the state value, the computer device can use the advantage function to calculate the advantage value GAE.

[0127] Among them, during the training process of the latest version of the game battle model, the action decision prediction sub-model in the latest version of the game battle model predicts the predicted action probability distribution based on the battle state characteristics.

[0128] Based on the above, the computer device trains the latest version of the game battle model based on the predicted action probability distribution, the initial predicted action probability distribution and the advantage value GAE, thereby obtaining a trained action decision prediction sub-model.

[0129] In the embodiment of the present application, the PPO algorithm can be used to train the action decision prediction sub-model in the latest version of the game battle model based on the predicted action probability distribution, the initial predicted action probability distribution and the advantage value GAE, so as to update the network parameters of the action decision prediction sub-model.

[0130] In addition, during the training process of the latest version of the game battle model, the state value sub-model in the latest version of the game battle model can also be trained.

[0131] Specifically, the state value sub-model in the latest version of the game battle model predicts the target state value based on the battle situation characteristics and battle character characteristics.

[0132] The embodiment of the present application calculates the average value between the reward score and the candidate state value, and uses the average value as the training label of the state value sub-model in the latest version of the game battle model. The state value sub-model in the latest version of the game battle model is trained based on the average value and the state value, thereby obtaining a trained state value sub-model.

[0133] Among them, the embodiment of the present application can use the mean square error loss function MSE to calculate the loss between the average value and the state value, so as to train the state value sub-model in the latest version of the game battle model based on the loss between the average value and the state value.

[0134] After obtaining the trained game battle model, the embodiment of the present application can store the trained game battle model in a game battle model set for the next round of training.

[0135] The embodiment of the present application can obtain a game battle model set, the game battle model set includes several game battle models, the game battle model includes an updated version of the game battle model obtained by iteratively training the initial game battle model; at least two target game battle models are selected from the game battle model set to perform game battles to generate battle status information between the current team and the enemy team, the target game battle models include the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set; the battle status information is extracted to obtain battle status features and position labels for the enemy team; the latest version of the game battle model is trained based on the battle status features and the position labels to obtain a trained game battle model. Since the embodiments of the present application can conduct a game battle with a target game battle model selected from a game battle model set, thereby generating battle status information, and training the game battle model based on the battle status information, the trained game battle model can be applied in the game battle. In this way, the trained game battle model can be used to predict the position of the enemy team, so that the trained game battle model can observe a game situation that is close to the global state of the game, thereby enhancing the accuracy of the trained game battle model in making game decisions and avoiding decision-making deviations caused by local game states.

[0136] This embodiment will be described from the perspective of a game battle device, which can be specifically integrated into a computer device, which can be a server or a terminal. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device, or other smart devices that can acquire data.

[0137] like Figure 8 As shown, the specific process of the game battle method is as follows:

[0138] S201. The computer device obtains a game battle model set.

[0139] Among them, the game battle model set includes several game battle models, and the game battle model includes an updated version of the game battle model obtained by iterative training of the initial game battle model.

[0140] S202: The computer device selects at least two target game battle models from the game battle model set to perform game battle, so as to generate battle status information between the current team and the enemy team.

[0141] Among them, the battle status information includes battle situation information and battle role information.

[0142] Among them, the target game battle model includes the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set.

[0143] In the embodiment of the present application, there are multiple ways to select at least two target game battle models from the game battle model set for game battle, as detailed below:

[0144] For example, in an embodiment of the present application, each game battle model has a model identifier, based on which the computer device can filter out a target game battle model from a set of game battle models based on the model identifier for game battle.

[0145] Among them, the computer device can randomly select a target game battle model from the game battle model set based on the model identifier to perform game battle.

[0146] For another example, an embodiment of the present application can filter out a target game battle model from a game battle model set based on a preset probability value. Specifically, a computer device obtains a preset probability value; when the preset probability value is greater than or equal to a preset threshold value, the latest version of the game battle model and the non-latest version of the game battle model are filtered out from the game battle model set to perform a game battle to generate battle status information.

[0147] The preset probability value can be set according to specific needs. The preset threshold value can be 0.9, but is not limited to 0.9, and the preset threshold value can be set according to specific needs.

[0148] S203. The computer device extracts the battle status information to obtain battle status features and a position tag for the enemy team.

[0149] The battle status information includes battle situation information and battle role information, and the battle status features include battle situation features and battle role features.

[0150] Among them, the embodiment of the present application can use the latest version of the game battle model to extract features of the battle situation information to obtain battle situation features; use the latest version of the game battle model to extract features of the battle character information to obtain battle character features; use the latest version of the game battle model to extract labels of the battle situation information and the battle character information to obtain position labels for the enemy team.

[0151] In the embodiment of the present application, before extracting the location tag of the enemy team, the computer device can pre-segment the global map of the game. For example, the computer device divides the global map into regions according to a preset distance to determine several regions of the global map; the computer device assigns a different number to each region. In this way, each region on the global map has a number, so that the location tag of the enemy team can be determined.

[0152] Based on this, the computer device determines the role of the enemy team based on the characteristics of the battle role; determines the target area on the global map where the character of the enemy team is located based on the global map characteristics in the battle situation characteristics and the role of the enemy team; determines the position tag of the enemy team based on the number of the target area, that is, the embodiment of the present application can use the number of the target area as the position tag of the enemy team.

[0153] S204. The computer device trains the latest version of the game battle model based on the battle state features and the location tags to obtain a trained game battle model.

[0154] The game battle model of the embodiment of the present application can be a model composed of multiple sub-models. The following will be explained using the latest version of the game battle model as an example.

[0155] The latest version of the game battle model includes an enemy position prediction sub-model and an action decision prediction sub-model. The computer device trains the latest version of the game battle model to obtain the trained game battle model as follows:

[0156] For example, based on the battle situation characteristics, battle role characteristics and position labels, the enemy position prediction sub-model is trained to obtain a trained enemy position prediction sub-model; based on the battle situation characteristics and battle role characteristics, the action decision prediction sub-model is trained to obtain a trained action decision prediction sub-model.

[0157] In the embodiment of the present application, the latest version of the game battle model also includes a state value sub-model. In contrast, the non-latest version of the game battle model includes a non-latest state value sub-model.

[0158] Based on this, the embodiment of the present application trains the action decision prediction sub-model based on the battle situation characteristics and the battle role characteristics, and the process of obtaining the trained action decision prediction sub-model can be as follows:

[0159] The computer device uses an action decision prediction sub-model to predict actions based on the battle situation characteristics and the battle role characteristics, and obtains the predicted action probability distribution of the current team for the battle state information; based on the battle situation characteristics and the battle role characteristics, a non-latest state value sub-model is used to predict value, and obtain the state value for the battle state information; based on the state value and the predicted action probability distribution, the action decision prediction sub-model is trained to obtain the trained action decision prediction sub-model.

[0160] Among them, the embodiment of the present application adopts the Actor-Critic algorithm to train the action decision prediction sub-model. The architecture of the Actor-Critic algorithm includes two neural network models, namely the Actor network model and the Critic network model. The Actor network model is used to generate strategies, and the Critic network model is used to evaluate the value of the strategies generated by the Actor network model.

[0161] In the embodiment of the present application, the action decision prediction sub-model is the Actor network model, and the state value sub-model is the Critic network model.

[0162] Similarly, the non-latest action decision prediction sub-model in the non-latest version of the game battle model is the Actor network model, and the non-latest state value sub-model in the non-latest version of the game battle model is the Critic network model.

[0163] Among them, the embodiments of the present application can use the enemy position prediction sub-model to predict the position of the enemy team based on the battle situation characteristics and the battle role characteristics, and obtain the predicted position of the enemy team; based on the predicted position of the enemy team, the battle situation characteristics and the battle role characteristics, use the action decision prediction sub-model to predict the action, and obtain the predicted action probability distribution of the current team for the battle status information.

[0164] In the embodiment of the present application, during the process of the target game battle model playing the game, the non-latest action decision prediction sub-model in the non-latest version of the game battle model predicts the initial predicted action probability distribution based on the battle situation characteristics and battle role characteristics of the battle state information, and selects the initial predicted action with the highest probability based on the initial predicted action probability distribution, thereby executing the initial predicted action, thus advancing the game process, and obtaining new battle state information; extracting features from the new battle state information, obtaining new battle situation characteristics and new battle role characteristics. Based on this, the non-latest action decision prediction sub-model calculates the reward score for the initial predicted action.

[0165] The state value includes the initial state value and the candidate state value. When the target game battle model is playing a game, the non-latest state value sub-model in the non-latest version of the game battle model calculates the initial state value based on the battle situation characteristics and battle character characteristics. The initial state value represents the value of the battle situation characteristics and battle character characteristics for action prediction.

[0166] The state value sub-model in the target game battle model calculates the candidate state value based on the new battle situation characteristics and the new battle role characteristics, and the candidate state value represents the value of the new battle situation characteristics and the new battle role characteristics for action prediction; the state value is generated based on the initial state value and the candidate state value. Then, based on the initial state value and the candidate state value in the state value, the computer device can use the advantage function to calculate the advantage value GAE.

[0167] Among them, during the training process of the latest version of the game battle model, the action decision prediction sub-model in the latest version of the game battle model predicts the predicted action probability distribution based on the characteristics of the battle situation and the characteristics of the battle characters.

[0168] Based on the above, the computer device trains the latest version of the game battle model based on the predicted action probability distribution, the initial predicted action probability distribution and the advantage value GAE, thereby obtaining a trained action decision prediction sub-model.

[0169] In the embodiment of the present application, the PPO algorithm can be used to update the network parameters of the action decision prediction sub-model in the latest version of the game battle model based on the predicted action probability distribution, the initial predicted action probability distribution and the advantage value GAE.

[0170] In an embodiment of the present application, in addition to using the PPO algorithm to perform reinforcement learning on the latest version of the game battle model, the A3C algorithm and the DDPG algorithm can also be used to perform reinforcement learning on the latest version of the game battle model.

[0171] S205. The computer device obtains battle status information between the current team and the enemy team in the current game.

[0172] Among them, the battle status information includes battle situation information and battle role information.

[0173] The trained game battle model of the embodiment of the present application can be applied to human-computer game battles, or machine-to-machine game battles.

[0174] For example, the trained game battle model is applied in a human-machine game battle, and the player and the trained game battle model are used as different teams to play the game. For example, the trained game battle model controls the current team, and the player controls the enemy team to play the game. The trained game battle model is applied in a game between machines, for example, two trained game battle models are used to control the current team and the enemy team respectively to play the game. In this way, battle status information can be generated.

[0175] S206: The computer device extracts features from the battle status information to obtain battle status features of the current game.

[0176] The embodiment of the present application can use the trained game battle model to extract features of the battle situation information to obtain the battle situation features of the current game; and use the trained game battle model to extract features of the battle character information to obtain the battle character features of the current game.

[0177] S207. The computer device predicts the position of the enemy team based on the battle status characteristics using the trained game battle model to determine the predicted position of the enemy team.

[0178] Among them, the post-training game battle model is the aforementioned post-training game battle model.

[0179] Among them, the trained game battle model includes a trained enemy position prediction sub-model. The computer device can use the trained enemy position prediction sub-model to predict the position of the enemy team based on the battle situation characteristics and battle role characteristics to determine the predicted position of the enemy team.

[0180] S208. The computer device predicts the action of the current team based on the predicted position of the enemy team using the trained game battle model, and controls the current team to execute the predicted action.

[0181] Among them, the post-training game battle model also includes a post-training action decision prediction sub-model. The computing device can use the post-training action decision prediction sub-model to predict the action of the current team based on the predicted position of the enemy team, the characteristics of the battle situation and the characteristics of the battle roles, and obtain the predicted action of the current team.

[0182] A game battle model set is obtained, the game battle model set includes several game battle models, and the game battle model includes an updated version of the game battle model obtained by iteratively training the initial game battle model; at least two target game battle models are selected from the game battle model set for game battle to generate battle status information between the current team and the enemy team, the target game battle models include the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set; the battle status information is extracted to obtain battle status features and position labels for the enemy team; the latest version of the game battle model is trained based on the battle status features and the position labels to obtain a trained game battle model. Since the embodiment of the present application can conduct a game battle with a target game battle model selected from a game battle model set, thereby generating battle status information, and training the game battle model based on the battle status information, the trained game battle model can be applied in the game battle. In this way, the trained game battle model can be used to predict the position of the enemy team, so that the trained game battle model can observe a game situation that is close to the global state of the game, thereby enhancing the accuracy of the trained game battle model in making game decisions and avoiding decision-making deviations caused by local game states.

[0183] In order to better implement the above method, an embodiment of the present application also provides a game battle model training device, which can be integrated into a computer device, such as a server or terminal, and the terminal may include a tablet computer, a laptop computer and / or a personal computer, etc.

[0184] For example, Fig. 9 As shown, the game battle model training device may include a first acquisition unit 301, a screening unit 302, a first extraction unit 303 and a training unit 304, as follows:

[0185] (1) a first acquisition unit 301;

[0186] The first acquisition unit 301 can be used to acquire a game battle model set, where the game battle model set includes several game battle models, and the game battle model includes an updated version of the game battle model obtained by iteratively training the initial game battle model.

[0187] (2) screening unit 302;

[0188] The screening unit 302 can be used to screen out at least two target game battle models from the game battle model set for game battle to generate battle status information between the current team and the enemy team. The target game battle models include the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set.

[0189] In some embodiments, the screening unit 302 can be specifically used to obtain a preset probability value; when the preset probability value is greater than or equal to a preset threshold, the latest version of the game battle model and the non-latest version of the game battle model are screened out from the game battle model set for game battle to generate battle status information.

[0190] (3) a first extraction unit 303;

[0191] The first extraction unit 303 can be used to extract the battle status information to obtain the battle status features and the position label for the enemy team.

[0192] (4) training unit 304;

[0193] The training unit 304 can train the latest version of the game battle model based on the battle state features and the position tags to obtain a trained game battle model.

[0194] In some embodiments, the latest version of the game battle model includes an enemy position prediction sub-model and an action decision prediction sub-model; the training unit 304 can be specifically used to train the enemy position prediction sub-model based on battle state characteristics and position labels to obtain a trained enemy position prediction sub-model; based on battle state characteristics, the action decision prediction sub-model is trained to obtain a trained action decision prediction sub-model.

[0195] In some embodiments, the non-latest version of the game battle model includes a non-latest state value sub-model; the training unit 304 can be specifically used to perform action prediction based on battle state characteristics using an action decision prediction sub-model to obtain the predicted action probability distribution of the current team for the battle state information; based on the battle state characteristics, a non-latest state value sub-model is used to perform value prediction to obtain the state value for the battle state information; based on the state value and the predicted action probability distribution, the action decision prediction sub-model is trained to obtain a trained action decision prediction sub-model.

[0196] In some embodiments, the training unit 304 can be specifically used to predict the position of the enemy team based on the battle status characteristics by using the enemy position prediction sub-model to obtain the predicted position of the enemy team; based on the predicted position of the enemy team and the battle status characteristics, an action decision prediction sub-model is used to predict the action to obtain the predicted action probability distribution of the current team for the battle status information.

[0197] As can be seen from the above, the first acquisition unit 301 of the embodiment of the present application can be used to obtain a game battle model set, the game battle model set includes several game battle models, and the game battle model includes an updated version of the game battle model obtained by iterative training of the initial game battle model; the screening unit 302 can be used to screen out at least two target game battle models from the game battle model set for game battle, so as to generate battle status information between the current team and the enemy team, the target game battle model includes the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set; the first extraction unit 303 can be used to extract the battle status information to obtain battle status features and position labels for the enemy team; the training unit 304 can be used to train the latest version of the game battle model based on the battle status features and the position labels to obtain the trained game battle model. Since the embodiments of the present application can conduct a game battle with a target game battle model selected from a game battle model set, thereby generating battle status information, and training the game battle model based on the battle status information, the trained game battle model can be applied in the game battle. In this way, the trained game battle model can be used to predict the position of the enemy team, so that the trained game battle model can observe a game situation that is close to the global state of the game, thereby enhancing the accuracy of the trained game battle model in making game decisions and avoiding decision-making deviations caused by local game states.

[0198] In order to better implement the above method, an embodiment of the present application also provides a game battle device, which can be integrated into a computer device, such as a server or a terminal, and the terminal may include a tablet computer, a laptop computer and / or a personal computer, etc.

[0199] For example, Fig.10 As shown, the game battle may include a second acquisition unit S301, a second extraction unit S302, a determination unit S303 and a prediction unit S304, as follows:

[0200] (1) a second acquisition unit S301;

[0201] The second acquisition unit S301 can be used to acquire the battle status information between the current team and the enemy team in the current game.

[0202] (2) second extraction unit S302;

[0203] The second extraction unit S302 may be used to extract features from the battle status information to obtain battle status features of the current game.

[0204] (3) Determine unit S303;

[0205] The determination unit S303 can be used to predict the position of the enemy team based on the battle state characteristics by using the trained game battle model to determine the predicted position of the enemy team. The trained game battle model is the aforementioned trained game battle model.

[0206] (4) prediction unit S304;

[0207] The prediction unit 304 can be used to predict the actions of the current team based on the predicted position of the enemy team using the trained game battle model, and control the current team to perform the predicted actions.

[0208] In some embodiments, the trained game battle model includes a trained enemy position prediction sub-model and a trained action decision prediction sub-model; the prediction unit 304 can be specifically used to predict the position of the enemy team based on the battle status characteristics using the trained enemy position prediction sub-model to determine the predicted position of the enemy team; based on the predicted position of the enemy team, the trained action decision prediction sub-model is used to predict the action of the current team.

[0209] As can be seen from the above, the second acquisition unit S301 of the embodiment of the present application can be used to obtain the battle status information of the current team and the enemy team in the current game; the second extraction unit S302 can be used to extract features of the battle status information to obtain the battle status features of the current game; the determination unit S303 can be used to predict the position of the enemy team based on the battle status features, using the trained game battle model to determine the predicted position of the enemy team; the prediction unit 304 can be used to predict the action of the current team based on the predicted position of the enemy team, using the trained game battle model, and control the current team to perform the predicted action. Since the embodiment of the present application can use the trained game battle model to predict the position of the enemy team, the trained game battle model can observe the game situation that is close to the global state of the game, thereby enhancing the accuracy of the trained game battle model in game decision-making and avoiding decision-making deviations caused by the local state of the game.

[0210] The present application also provides a computer device, such as Fig.11 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0211] The computer device may include one or more processing core processors 401, one or more computer-readable computer-readable storage media memory 402, a power supply 403, an input unit 404 and other components. Those skilled in the art will understand that Fig.11 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0212] The processor 401 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 402 and calling data stored in the memory 402, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and computer programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0213] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0214] The computer device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0215] The computer device may further include an input unit 404, which may be used to receive input digital or character information communications and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0216] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the computer device will load the executable files corresponding to the processes of one or more computer programs into the memory 402 according to the following instructions, and the processor 401 will run the computer programs stored in the memory 402, thereby realizing various functions, as follows:

[0217] A game battle model set is obtained, the game battle model set includes several game battle models, and the game battle model includes an updated version of the game battle model obtained by iteratively training the initial game battle model; at least two target game battle models are selected from the game battle model set for game battle to generate battle status information between the current team and the enemy team, the target game battle models include the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set; the battle status information is extracted to obtain battle status features and position labels for the enemy team; the latest version of the game battle model is trained based on the battle status features and the position labels to obtain a trained game battle model.

[0218] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0219] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0220] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute any game battle model training method provided in the embodiment of the present application.

[0221] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0222] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0223] Since the instructions stored in the computer-readable storage medium can execute the steps in any game battle model training method provided in the embodiments of the present application, the beneficial effects that can be achieved by any game battle model training method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0224] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations provided in the above embodiments.

[0225] The above is a detailed introduction to a game battle model training method, a game battle method and related devices provided in the embodiments of the present application, wherein the related devices include a game battle model training device, a game battle device, a computer device and a computer-readable storage medium. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A game battle model training method, characterized in that: include: Acquire a game battle model set, wherein the game battle model set includes a plurality of game battle models, and the game battle model includes an updated version of a game battle model obtained by iteratively training an initial game battle model; At least two target game battle models are selected from the game battle model set to perform game battles, so as to generate battle status information of the current team and the enemy team, wherein the target game battle models include the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set; the latest version of the game battle model includes an enemy position prediction sub-model and an action decision prediction sub-model; the non-latest version of the game battle model includes a non-latest status value sub-model; Extracting the battle status information to obtain battle status features and a position label for the enemy team; The latest version of the game battle model is trained based on the battle state feature and the position label to obtain a trained game battle model, including: based on the battle state feature and the position label, the enemy position prediction sub-model is trained to obtain a trained enemy position prediction sub-model; Based on the battle status characteristics, the action decision prediction sub-model is used to perform action prediction to obtain the predicted action probability distribution of the current team for the battle status information; based on the battle status characteristics, the non-latest state value sub-model is used to perform value prediction to obtain the state value for the battle status information; based on the state value and the predicted action probability distribution, the action decision prediction sub-model is trained to obtain the trained action decision prediction sub-model.

2. The game battle model training method according to claim 1, characterized in that: Based on the battle status feature, the action decision prediction sub-model is used to perform action prediction to obtain the predicted action probability distribution of the current team for the battle status information, including: Based on the battle status characteristics, the enemy position prediction sub-model is used to predict the position of the enemy team to obtain the predicted position of the enemy team; Based on the predicted position of the enemy team and the battle status characteristics, the action decision prediction sub-model is used to perform action prediction to obtain the predicted action probability distribution of the current team for the battle status information.

3. The game battle model training method according to claim 1, characterized in that: The step of selecting at least two target game battle models from the game battle model set to perform game battles to generate battle status information between the current team and the enemy team includes: Get the preset probability value; When the preset probability value is greater than or equal to a preset threshold, the latest version game battle model and the non-latest version game battle model are selected from the game battle model set to perform game battles to generate battle status information.

4. A game battle method, characterized in that: include: Get the battle status information between the current team and the enemy team in the current game; Extracting features from the battle status information to obtain battle status features of the current game; Based on the battle status characteristics, a trained game battle model is used to predict the position of the enemy team to determine the predicted position of the enemy team, wherein the trained game battle model is a trained game battle model as described in any one of claims 1 to 3; Based on the predicted position of the enemy team, the trained game battle model is used to predict the actions of the current team, and the current team is controlled to execute the predicted actions.

5. The game battle method according to claim 4, characterized in that: The post-training game battle model includes a post-training enemy position prediction sub-model and a post-training action decision prediction sub-model; based on the battle state characteristics, the post-training game battle model is used to predict the position of the enemy team to determine the predicted position of the enemy team, including: Based on the battle status characteristics, the trained enemy position prediction sub-model is used to predict the position of the enemy team to determine the predicted position of the enemy team; The method of using the trained game battle model to predict the actions of the current team based on the predicted position of the enemy team includes: using the trained action decision prediction sub-model to predict the actions of the current team based on the predicted position of the enemy team.

6. A game battle model training device, characterized in that: include: A first acquisition unit is used to acquire a game battle model set, wherein the game battle model set includes a plurality of game battle models, and the game battle model includes an updated version of the game battle model obtained by iteratively training the initial game battle model; A screening unit is used to screen out at least two target game battle models from the game battle model set for game battle, so as to generate battle status information of the current team and the enemy team, wherein the target game battle model includes the latest version of the game battle model in the game battle model set and the non-latest version of the game battle model in the game battle model set; the latest version of the game battle model includes an enemy position prediction sub-model and an action decision prediction sub-model; the non-latest version of the game battle model includes a non-latest status value sub-model; A first extraction unit is used to extract the battle status information to obtain battle status features and a position label for the enemy team; A training unit, used to train the latest version of the game battle model based on the battle state feature and the position label to obtain a trained game battle model, including: training the enemy position prediction sub-model based on the battle state feature and the position label to obtain a trained enemy position prediction sub-model; Based on the battle status characteristics, the action decision prediction sub-model is used to perform action prediction to obtain the predicted action probability distribution of the current team for the battle status information; based on the battle status characteristics, the non-latest state value sub-model is used to perform value prediction to obtain the state value for the battle status information; based on the state value and the predicted action probability distribution, the action decision prediction sub-model is trained to obtain the trained action decision prediction sub-model.

7. A computer device, characterized in that: It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the game battle model training method described in any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the game battle model training method described in any one of claims 1 to 3.

Images