Ai model training method, action control method, server, and storage medium

Abstract

The application discloses an AI model training method, an action control method, a server and a computer readable storage medium. The model training method comprises the following steps: obtaining a training sample of a game character, wherein the training sample comprises offline data of the game character and first-level labels and second-level labels, the first-level labels represent action information performed by the game character, and the second-level labels represent direction information of the action performed by the game character; performing feature extraction on the training sample, obtaining action features, inputting output generated by inputting the action features into a preset multi-layer neural network into a first branch, obtaining predicted action information of the game character and direction information of a predicted action, calculating a loss function of the predicted action information and the first-level labels and the predicted action direction information and the second-level labels, and stopping training if the loss function meets a preset convergence condition, thereby obtaining an AI model. Therefore, the humanization of the AI model is improved through fine modeling of the game character.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to an AI model training method, an action control method, a server and a computer readable storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence (AI) technology, the application of AI models trained by artificial intelligence in games has attracted widespread attention.

[0003] At present, for the application scene related to a Multiplayer Online Battle Arena (MOBA) game, modeling is generally performed based on a large amount of game battle data of each game character. The large amount of data causes low training efficiency and occupies a large amount of computing resources, and the human-likeness of the trained AI model also needs to be improved, which affects the game experience of users to a certain extent. Therefore, how to use a small amount of data to perform more fine modeling on the AI model of the game character in the game has become a problem to be solved. SUMMARY

[0004] The AI model training method, the action control method, the server and the computer readable storage medium disclosed by the present application solve the problem of low human-likeness of the AI model of a game character in a MOBA game.

[0005] In a first aspect, the present application provides an AI model training method, comprising:

[0006] obtaining a training sample of a game character from a game battle, wherein the training sample comprises offline data of the game character and corresponding labels, the labels comprise a first-level label and a second-level label, the first-level label represents action information of the game character, and the second-level label represents direction information of the action of the game character;

[0007] performing feature extraction on the training sample to obtain action features;

[0008] inputting the action features of the game character into a preset multi-layer neural network, inputting an output of the preset multi-layer neural network into a first branch, and obtaining predicted action information and direction information of the predicted action of the game character according to the output of the preset multi-layer neural network and a preset weight in the first branch, and calculating a loss function of the predicted action information and the first-level label and the predicted action direction information and the second-level label;

[0009] stopping training if the loss function meets a preset convergence condition, and obtaining the AI model.

[0010] In a second aspect, the application provides an action control method, comprising:

[0011] obtaining battle real-time data of an agent in a game battle;

[0012] inputting the battle real-time data of the agent into an AI model to perform feature extraction, to obtain action features, the AI model being obtained by training the AI model training method in any one of claims 1-7;

[0013] obtaining a second weight of the agent, inputting the second weight into the AI model, and outputting action information of the game character according to the second weight and the action features, to make the game character perform an action corresponding to the first action information.

[0014] In a third aspect, the application also provides a server, characterized in that the server comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, the memory storing a policy model, wherein the computer program is executed by the processor to implement the AI model training method as described above, or to implement the action control method as described above.

[0015] In a fourth aspect, the application also provides a computer readable storage medium for storing a computer program, the computer program being executed by a processor to make the processor implement the AI model training method as described above, or to implement the action control method as described above.

[0016] The embodiment of the application provides an AI model training method, an action control method, a server and a computer readable storage medium. The AI model training method extracts the action features of a game character from offline data features of the game character, inputs the action features of the game character into a preset multi-layer neural network, inputs the output of the preset multi-layer neural network into a first branch, obtains the predicted action information and the direction information of the predicted action of the game character according to the output of the preset multi-layer neural network and a preset weight, calculates the loss function of the predicted action information and the first-level label and the loss function of the direction information of the predicted action and the second-level label, and completes the training of the AI model if the loss function meets a preset convergence condition. The action features of the game character are obtained by extracting the offline data features of the game character, the predicted action information and the direction information of the predicted action of the game character are obtained by inputting the action features of the game character into a preset multi-layer neural network, the loss function of the predicted action information and the first-level label and the loss function of the direction information of the predicted action and the second-level label are calculated, and the training of the AI model is completed if the loss function meets a preset convergence condition. The AI model training method provided by the application realizes the weight learning shared by multiple heroes by uniformly modeling the hero skill label, so as to predict the action information and the direction information of the action of the game character, the heroes only need a small amount of data to train a good effect, and the personification of the constructed AI model is improved.

[0017] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is a step schematic flow chart of an AI model training method provided by the embodiment of the application;

[0020] Figure 2 is a step schematic flow chart of a game character offline data determination method provided by the embodiment of the application;

[0021] Figure 3 is a schematic diagram of a game character offline data classification method provided by the embodiment of the application;

[0022] Figure 4 is a step schematic flow chart of a game character offline data determination method provided by the embodiment of the application;

[0023] Figure 5 FIG. 1 is a schematic diagram of a method for obtaining action features of a game character according to an embodiment of the present application;

[0024] Figure 6 FIG. 2 is a schematic diagram of a method for obtaining prediction action information and direction information of a preset action according to an embodiment of the present application;

[0025] Figure 7 FIG. 3 is a schematic diagram of a method for constructing an AI model according to an embodiment of the present application;

[0026] Figure 8 FIG. 4 is a schematic diagram of a method for obtaining prediction action information and direction information of a preset action according to an embodiment of the present application;

[0027] Figure 9 FIG. 5 is a schematic diagram of a method for obtaining a first weight according to an embodiment of the present application;

[0028] Figure 10 FIG. 6 is a schematic diagram of another method for obtaining prediction action information and direction information of a preset action according to an embodiment of the present application;

[0029] Figure 11 FIG. 7 is a schematic diagram of another method for obtaining a first weight of a game character according to an embodiment of the present application;

[0030] Figure 12 FIG. 8 is a schematic diagram of a method for controlling action according to an embodiment of the present application;

[0031] Figure 13 FIG. 9 is a schematic block diagram of a server according to an embodiment of the present application.

[0032] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0034] The flowcharts shown in the drawings are only exemplary, and do not necessarily include all the contents and operations / steps, nor do they have to be executed in the order described. For example, some operations / steps can be further divided, combined or partially merged, so the actual execution order may be changed according to the actual situation.

[0035] It should be understood that the terms used in this specification of the application are only for the purpose of describing particular embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an" and "the" are intended to include plural forms as well, unless the context clearly indicates otherwise.

[0036] It should be understood that, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the terms "first", "second", etc. are used to distinguish the same items or similar items with basically the same function and role. For example, the first identification model and the second identification model are only used to distinguish different callback functions, and do not limit the order. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. also do not necessarily mean different.

[0037] It should also be understood that the term "and / or" used in the specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0038] In order to facilitate the understanding of the embodiments of the present application, some glossaries involved in the embodiments of the present application are briefly explained as follows.

[0039] 1. MOBA: MOBA games generally need to purchase equipment in the battle, players are usually divided into two teams, two teams compete with each other in the scattered game map, and each player controls the selected character through the interface. MOBA game users usually do not need to operate the organization units such as buildings, resources and trained troops commonly seen in Action Real-Time Strategy (ARTS) games such as "StarCraft", and players only need to control their selected characters.

[0040] 2. Action information: the action of the game character can be divided into three categories of stop, move and skill. When the action of the game character is stop, the direction information of the game character action is 0; when the action of the game character is move, the direction information of the game character action is the moving direction; when the action of the game character is skill, the direction information of the game character action is divided into the category of skill and the direction of skill release. The category of skill is generally divided into the following four categories:

[0041] 1). Click to release, button type skill;

[0042] 2). Drag the skill to release in a certain direction, direction type skill;

[0043] 3). Drag skills are released at a certain position, position-type skills;

[0044] 4). Skills are released on a certain target on the battlefield, target-type skills.

[0045] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following examples and features in the examples can be combined with each other without conflict.

[0046] At present, for the application scenario involving MOBA games, a large amount of game battle data of each game character is generally modeled, and independent model training is performed on each established model. The trained models are used to control the actions of the corresponding agents. There may be a situation of excessive training data, which may cause a large amount of computing resources to be occupied, and the humanization of the trained AI model needs to be improved.

[0047] To solve the above problems, embodiments of the present application provide an AI model training method, an action control method, a server and a computer readable storage medium, which can improve the accuracy and reliability of the AI model output prediction action information. The AI model training method and the action control method can be applied to a server, which can be a single server or a server cluster composed of multiple servers.

[0048] Please refer to Figure 1 , Figure 1 is a step schematic flowchart of the AI model training method provided by an embodiment of the present application.

[0049] As Figure 1 shown, the model training method specifically includes steps S101 to S104.

[0050] S101, obtaining a training sample of a game character from a game battle, the training sample including offline data of the game character and corresponding labels, the labels including first-level labels and second-level labels, the first-level labels representing action information performed by the game character, and the second-level labels representing direction information of the action performed by the game character.

[0051] In each training process, the offline data of a game character and its corresponding labels are randomly obtained as a training sample. Assuming that there are 100 offline data of game characters in the game, one hero's data is randomly selected for training each time, and in each training sample, in addition to the offline data of the game character obtained from the game battle, there are first-level labels and second-level labels set by the offline data of the game character. The first-level labels are the action information performed by the game character in the offline data corresponding to the labels, and the second-level labels are the direction information corresponding to the action information.

[0052] For example, the action information of the game character is divided into three categories: stop, move and skill. When the first-level label of the game character corresponds to the information of stop, the first-level label is 1 and the second-level label is 0. When the first-level label of the game character corresponds to the information of move, the first-level label is 2 and the second-level label is the moving direction of the game character. Assuming that the moving direction of the game character is set to 10 directions, the second-level label is any value from 1 to 10. When the first-level label of the game character corresponds to the information of skill, the first-level label is 3 and the second-level label is the type of skill and the direction of skill release. It is assumed in the application that the game character has four types of skills, which are 1-button type, 2-direction type, 3-position type and 4-target type. The direction of skill release is also set to 10, for example, when the second-level label is (3, 4), it means that the game character releases the position type skill in the fourth direction.

[0053] In some embodiments, referring to Figure 2 , Figure 2 is a step schematic flowchart of a game character offline data determination method provided by the application. The target table determination method includes steps S101a to S101b.

[0054] S101a, obtaining a first score of the offline data of the game character, the first score being used to represent the quality of the offline data of the game character.

[0055] S101b, obtaining the offline data of the game character whose first score is greater than or equal to a preset score.

[0056] The system scores the performance of the game character in each game match after the game match. It is assumed that the score range of the game character in each game match is 0-10 points, and the preset score is set to 8 points. The offline data of all game characters whose scores are greater than or equal to 8 points is screened and obtained. By screening the offline data of high quality, even if the data amount of the offline data of the game character is small at this time, the data of low-quality game players will not be introduced to cause strange behavior of the AI model after training.

[0057] In some embodiments, as Figure 3 shown, Figure 3is a schematic diagram of a game role offline data classification method provided by an embodiment of the present application. By obtaining the offline data of a game role corresponding to rating information, the rating information is used to represent the offline data of the game role in a battle level. The offline data of the game role with the same rating information is merged. By merging the offline data of the game role, the AI model that is built can learn the operation of the game player in different game levels, and the predicted action information is more consistent with the rating corresponding to the game battle, overcoming the traditional AI model training that is for high-level game players, and improving the experience of game players in different levels.

[0058] As shown in the example, Figure 3 Assuming that the rating of the MOBA game is divided into bronze, silver, gold, platinum, diamond, master and king, the offline data of the game role with the same rating is merged, and the AI model in the training process can also be built to correspond to different ratings, that is, the AI model corresponding to the bronze rating is only trained for the offline data of the game role in the bronze rating. When called, the corresponding AI model is called according to the rating information of the game battle, so that the intelligent agent controlled by the AI model can perfectly integrate into the rating.

[0059] In some embodiments, please refer to Figure 4 , Figure 4 is a step schematic flow chart of a game role offline data determination method provided by an embodiment of the present application. The target table determination method includes steps S101c to S101d.

[0060] S101c, the offline data of all game roles in the game battle and the corresponding labels are sent to the data loader, so that the data loader batch processes the offline data of all game roles.

[0061] S101d, obtaining the offline data of all game roles and the corresponding labels after batch processing by the data loader.

[0062] After the offline data of the game role is labeled, the filtered offline data is converted into vector data, and the vector data and the corresponding labels are sent to the data loader. The data loader batch processes (Batch processing) the vector data and the corresponding labels, and obtains the vector data and the corresponding labels after batch processing.

[0063] S102, feature extraction is performed on the training sample to obtain action features.

[0064] Specifically, by extracting the action features of the training samples, the purpose of feature extraction is to enable the AI model to understand the actions of the game characters, and to adopt the same learning method as the player in controlling a game character, and to have the ability to judge the timing of skill release through learning the description of the skill.

[0065] In some embodiments, as Figure 5 shown, Figure 5 is a game character action feature acquisition method schematic diagram provided by the embodiment of the application, by searching the preset action feature keywords in the offline data of the game character, the action features of the game character are acquired.

[0066] For example, the preset action feature keywords are displacement skills, skill base damage, skill damage bonus, skill range, searching the information corresponding to the preset action feature keywords in the offline data of the game character can extract the corresponding action features of the game character, for example, assuming that the game character has two skills, after the offline data of a game character is extracted, the action features obtained are: 1 skill is a displacement skill, the skill base damage is 10, the skill damage bonus is 15%, the skill range is 300 yards, 2 skill is a directional skill, the skill base damage is 20, the skill damage bonus is 30%, the skill range is 100 yards, by extracting the offline data of the game character, the AI can better understand the skill and better predict the action information of the game character.

[0067] S103, input the action features of the game character into a preset multi-layer neural network, the output of the preset multi-layer neural network is input into a first branch, the first branch obtains the predicted action information and the predicted action direction information of the game character according to the output of the preset multi-layer neural network and a preset weight, and calculates the loss function of the predicted action information and the first level label and the predicted action direction information and the second level label.

[0068] Specifically, the multi-layer neural network is an algorithm mathematical model that simulates the behavior characteristics of animal neural networks and performs distributed parallel information processing. The neural network relies on the complexity of the system to adjust the weight parameters of the mutual connection between a large number of neuron nodes inside, so as to achieve the effect of fitting the output and the expected result. The multi-layer neural network model for predicting the action information and the direction information of the action of the game character is the AI model generated by the application, wherein the input is the action features of the game character, and the output is the predicted action information and the direction information of the action.

[0069] Specifically, as Figure 6 shown, Figure 6is a schematic diagram of an acquisition method for predicting action information and direction information of a preset action provided by an embodiment of the present application. The first branch includes a full connection layer and a first weight. The output of the preset multi-layer neural network is input to the full connection layer, and the full connection layer and the first branch are used to generate the predicted action information and the direction information of the preset action. All game characters are trained using the same weight, so as to regress the data of all heroes and ensure that the heroes with a small amount of data also have good training effect.

[0070] The action features in the training sample data are input to the AI model to be constructed for model training, and a constructed AI model is obtained. When predicting the execution action of the game character, the execution action of the game character can be determined according to the prediction result output by the AI model.

[0071] In some embodiments, in order to improve the model construction efficiency, a shallow neural network model can be selected for the first identification model. The shallow neural network model is divided into three layers, namely an input layer, a hidden layer and an output layer, as shown in Figure 7 Figure 7 is a step schematic diagram of an AI model construction method provided by an embodiment of the present application. It is assumed that five feature parameters are input, including whether the skill is a displacement skill, skill base damage, skill damage addition, skill range, and whether the skill is physical damage or magic damage, and the five parameters need to be normalized before input.

[0072] In some embodiments, in order to improve the training efficiency, an adaptive moment estimation (Adam) optimizer is used for optimization during the training process. The gradient of the loss function is calculated in each iteration, and then the parameters are updated to improve the training quality.

[0073] In some embodiments, the loss functions of the predicted action information and the first-level label, and the predicted action direction information and the second-level label are calculated by using cross-entropy loss (Cross-Entropy), and the formula is as follows:

[0074] L = -(y log(p) + (1-y) log(1-p))

[0075] In the formula, L represents the loss function value, y represents the label corresponding to the training sample, and p represents the information predicted by the multi-layer neural network. By calculating the value of the cross-entropy loss function of the predicted information of the training sample and the label, the accuracy of the prediction information of the preset multi-layer neural network can be well reflected.

[0076] ​It should be noted that the construction of the AI model can be completed only when the predicted action information and the loss function value L1 of the first level label and the predicted action direction information and the loss function value L2 of the second level label both meet the convergence condition.

[0077] In some embodiments, after the action features of the game character are input into the preset multi-layer neural network, please refer to Figure 8 , Figure 8 is a step schematic flow chart of an acquisition method of predicted action information and preset action direction information provided by the embodiments of the present application. The target table determination method comprises steps S103a to S103b.

[0078] S103a, acquiring a first weight of the game character, and inputting an output of the preset multi-layer neural network into a second branch;

[0079] S103b, the second branch acquires the predicted action information and the direction information of the preset action of the game character according to the first weight of the game character and the output of the preset multi-layer neural network.

[0080] By acquiring the corresponding first weight of the game character, the output of the preset multi-layer neural network is input into the fully connected layer of the second branch. The second branch can acquire the predicted action information and the direction information of the preset action of the game character according to the input of the fully connected layer and the first weight. By training the game character based on its corresponding first weight, the training effect of the game character can be further improved in the case of sufficient data.

[0081] In some embodiments, after the action features of the game character are input into the preset multi-layer neural network, please refer to Figure 9 , Figure 9 is a step schematic flow chart of a first weight acquisition method provided by the embodiments of the present application. The target table determination method comprises steps S103a1 to S103a2.

[0082] S103a1, querying a corresponding preset identifier in a weight dictionary according to the identity identifier of the game character.

[0083] S103a2, acquiring a preset weight corresponding to the preset identifier, and the preset weight is used as the first weight of the game character.

[0084] The weight dictionary comprises preset identifiers and corresponding preset weights. By querying the identity identifier of the game character with the preset identifiers in the weight dictionary first, the preset weight corresponding to the queried preset identifier when the identity identifier is consistent with the preset identifier can be used as the first weight corresponding to the game character.

[0085] For example,Figure 10 As shown, Figure 10 is a schematic diagram of another method for acquiring predicted action information and direction information of a preset action provided by an embodiment of the present application. After the action characteristics of a game character are input into a multi-layer neural network, the input is output to a full connection layer of a second branch. Then, the first weight of the game character is obtained by querying the weight dictionary according to the identity of the game character. The predicted action information and the direction information of the predicted action of the game character are obtained. The combination of the weight and the output of the multi-layer neural network can more accurately complete the training of the AI model on each game character training sample. The AI model constructed has stronger humanization.

[0086] For example, the identity of the game character can be the volume of the game character. Because in actual game battles, the action of the game character is directly affected by its volume, it is assumed that there are 100 game characters in the current game. When the weight dictionary is prepared, the preset identity is arranged in order according to the volume of the game character. At the same time, the preset weight can also be arranged according to the arrangement order of the preset identity. The preparation method of the weight dictionary provided can quickly obtain the weight corresponding to the game character in each training process, and can also prevent the training of different characters from affecting each other.

[0087] For example, please refer to Figure 11 , Figure 11 is a step schematic flow chart of another method for acquiring the first weight of a game character provided by an embodiment of the present application. The target table determination method includes steps S103c to S103d.

[0088] S103c, when the identity of the game character cannot be queried in the weight dictionary, the preset identity with the highest similarity to the identity of the game character is queried in the weight dictionary according to the identity of the game character;

[0089] S103d, acquiring the preset weight corresponding to the preset identity, the preset weight as the first weight of the game character.

[0090] When a new game character appears in the game, the AI model training method provided by the embodiment of the present application can greatly reduce the data amount of the training of the new game character, and can also enable the AI model to learn and complete the training of the new game character faster, even if the data amount of the game battle at this time is small.

[0091] For example, assuming that there are originally 100 game characters, when the 101st game character is developed, the volume of the game character is taken as the identity of the game character, a preset identity that is closest to the volume of the new game character is queried in the weight dictionary, and the preset weight corresponding to the queried preset identity is the first weight of the new game character.

[0092] S104, if the loss function meets the preset convergence condition, stop training, and obtain the AI model.

[0093] Specifically, the preset convergence condition can be that the loss function value of the predicted action information and the direction information of the predicted action is less than a preset value, or that the loss function value of the predicted action information and the direction information of the predicted action remains in a preset interval and fluctuates. When the loss function meets the preset convergence condition, the training is stopped, and the constructed AI model is obtained.

[0094] The embodiment of the present application provides an AI model training method. The action features of the game characters are obtained by extracting the offline data features of the game characters, the action features of the game characters are input into a preset multi-layer neural network, and the output of the preset multi-layer neural network is input into a first branch. The first branch obtains the predicted action information and the direction information of the predicted action of the game character according to the output of the preset multi-layer neural network and the preset weight, calculates the loss function of the predicted action information and the first-level label and the predicted action direction information and the second-level label, and if the loss function meets the preset convergence condition, the training of the AI model is completed. The AI model training method provided by the present application can realize the weight learning shared by multiple heroes by uniformly modeling the hero skill labels, so as to predict the action information and the direction information of the action of the game character, so that each hero can be trained to a good effect with a small amount of data, and the personification of the constructed AI model is improved.

[0095] The embodiment of the present application also provides an action control method. The model can be the AI model described in the above embodiment, and the calling method of the model can be applied to a server to realize reliable and accurate AI simulation of the agent by calling the trained strategy model. The server can be a single server or a server cluster composed of multiple servers.

[0096] Please refer to Figure 12 , Figure 12 The steps of the action control method provided by the embodiment of the present application are shown in the flowchart.

[0097] As Figure 12 shown, the calling method of the model includes steps S201 to S203.

[0098] S201, obtaining the real-time data of the agent in the game battle.

[0099] Specifically, when the AI model training is completed, real-time data of a game battle in which the game character controlled by the agent is located is obtained in a real-time game, and the completed AI model based on the real-time data can output a control strategy of the game character by the agent.

[0100] In some embodiments, the obtained game battle real-time data further includes rating information of the game battle in which the agent is located, and the corresponding AI model matched through the rating information can make the action information of the agent match the rating information, thereby avoiding the experience of the remaining real players in the game battle from being reduced.

[0101] S202, input the game battle real-time data of the agent into the AI model for feature extraction to obtain action features, and the AI model is obtained by training the AI model training method in any one of claims 1-7.

[0102] Specifically, by extracting features from the real-time game data of the agent, the action features of the game character controlled by the agent are obtained as the input of the completed AI model, and the AI model can be applied to the real-time game.

[0103] In some embodiments, assuming that the action features to be extracted of the game character are displacement skills, skill base damage, skill damage bonus, and skill range, the information corresponding to the action features to be extracted is searched in the real-time game of the game character controlled by the agent, and the corresponding action features of the game character are extracted and input into the AI model.

[0104] S203, the AI model outputs the action information and the direction information of the action of the game character according to the action features, so that the game character performs the action information according to the direction information of the action.

[0105] When the AI model receives the action features of the game character controlled by the agent, it can output the action information and the direction information of the action of the game character, and through the action information and the direction information of the action, the agent can be controlled in real time in the game battle to perform the corresponding action information according to the direction information of the action.

[0106] The embodiments of the present application provide an action control method, by inputting the real-time game data of the game character controlled by the agent into the completed AI model, after feature extraction, the action information and the direction information of the action of the game character are output, and the game character controlled by the agent performs the corresponding action information according to the direction information of the action. The action control method provided by the present application can reduce the difference between the game character controlled by the agent and the real player, and improve the game experience of the player.

[0107] Please refer to Figure 13 ,Figure 13 A schematic block diagram of a server provided by an embodiment of the present application.

[0108] As shown in Figure 13 The server can include a processor, a memory and a network interface. The processor, the memory and the network interface are connected through a system bus, such as an I2C (Inter-integrated Circuit) bus.

[0109] Specifically, the processor can be a microcontroller unit (MCU), a central processing unit (CPU) or a digital signal processor (DSP), etc.

[0110] Specifically, the memory can be a flash chip, a read-only memory (ROM) disk, an optical disk, a U disk or a mobile hard disk, etc.

[0111] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that Figure 13 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the server to which the scheme of the present application is applied. The specific server can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0112] The processor is configured to run a computer program stored in the memory and implement the following steps when executing the computer program:

[0113] obtaining a training sample of a game character from a game battle, the training sample including offline data of the game character and corresponding labels, the labels including a first-level label and a second-level label, the first-level label representing action information performed by the game character, and the second-level label representing direction information of the action performed by the game character;

[0114] performing feature extraction on the training sample to obtain action features;

[0115] inputting the action features of the game character into a preset multi-layer neural network to obtain predicted action information and predicted action direction information of the game character, and calculating a loss function of the predicted action information and the first-level label and the predicted action direction information and the second-level label;

[0116] If the loss function meets a preset convergence condition, the training is stopped, and the AI model is obtained.

[0117] In some embodiments, before the processor obtains the training sample of the game role from the game battle, the processor further implements:

[0118] obtaining a first score of the offline data of the game role, the first score being used to represent the quality of the offline data of the game role;

[0119] obtaining the offline data of the game role with the first score greater than or equal to a preset score.

[0120] In some embodiments, before the processor obtains the training sample of the game role from the game battle, the processor further implements:

[0121] obtaining rating information corresponding to the offline data of the game role, the rating information being used to represent the battle level of the offline data of the game role, and merging the offline data of the game roles with the same rating information.

[0122] In some embodiments, before the processor obtains the training sample of the game role from the game battle, the processor further implements:

[0123] sending the offline data of all game roles in the game battle and corresponding labels to the data loader, so that the data loader batch processes the offline data of all game roles;

[0124] obtaining the offline data of all game roles and corresponding labels after batch processing by the data loader.

[0125] In some embodiments, before the processor obtains the predicted action information and the direction information of the predicted action of the game role, the processor further implements obtaining a first weight of the game role, and obtaining the predicted action information and the direction information of the predicted action of the game role according to the first weight of the game role and the output of the preset multi-layer neural network; in the method of obtaining the first weight, the processor specifically implements:

[0126] querying a corresponding preset identifier in a weight dictionary according to the identity identifier of the game role;

[0127] obtaining a preset weight corresponding to the preset identifier, the preset weight being used as the first weight of the game role.

[0128] In some embodiments, in the method of obtaining the first weight, the processor specifically implements:

[0129] When the identity of the game character cannot be queried in the weight dictionary to obtain the preset identity, searching for a preset identity with the highest similarity to the identity of the game character in the weight dictionary;

[0130] Obtaining the preset weight corresponding to the preset identity, the preset weight as the first weight of the game character.

[0131] In some embodiments, the processor implements the feature extraction on the training sample, specifically:

[0132] Searching for a preset action feature keyword in the offline data of the game character to obtain the action feature of the game character.

[0133] In some embodiments, the processor is configured to run a computer program stored in the memory, and implement the following steps when executing the computer program:

[0134] Obtaining the real-time data of the game character in the game battle;

[0135] Inputting the real-time data of the game character in the game battle into an AI model to extract features and obtain an action feature, the AI model being trained by any one of the AI model training methods provided in the embodiments;

[0136] The AI model outputs the action information and the direction information of the action of the game character according to the action feature, so that the game character performs the action information according to the direction information of the action.

[0137] It should be noted that, for the convenience and brevity of description, the specific working process of the server described above can refer to the corresponding process in the AI model training method and / or the action control method embodiments described above, which will not be described here.

[0138] The embodiments of the present application also provide a computer readable storage medium storing a computer program, the computer program including program instructions, and the processor executes the program instructions to implement the steps of the AI model training method and / or the action control method provided in the above embodiments. For example, the computer program is loaded by the processor, and the following steps can be executed:

[0139] Obtaining a training sample of a game character from a game battle, the training sample including offline data of the game character and corresponding labels, the labels including a first-level label and a second-level label, the first-level label representing action information of the game character, and the second-level label representing direction information of the action of the game character;

[0140] perform feature extraction on the training samples to obtain action features;

[0141] input the action features of the game character into a preset multi-layer neural network to obtain predicted action information and direction information of the predicted action of the game character, and calculate a loss function of the predicted action information and the first-level label and the predicted action direction information and the second-level label;

[0142] If the loss function meets a preset convergence condition, stop training to obtain the AI model.

[0143] The specific implementation of each operation can be referred to the foregoing embodiments, which will not be described here.

[0144] The computer readable storage medium can be an internal storage unit of the server, for example, a hard disk or a memory of the server. The computer readable storage medium can also be an external storage device of the server, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0145] Due to the computer program stored in the computer readable storage medium, any AI model training method and / or action control method provided by the embodiments of the present application can be executed, and thus the beneficial effects of any AI model training method and / or action control method provided by the embodiments of the present application can be achieved. Details are described in the foregoing embodiments, which will not be described here.

[0146] The above sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An AI model training method, characterized in that, include: The training sample of a game character is obtained from the game battle. The training sample includes the offline data of the game character and the corresponding label. The label includes a first-level label and a second-level label. The first-level label represents the action information performed by the game character, and the second-level label represents the direction information of the action performed by the game character. Feature extraction is performed on the training samples to obtain action features; The motion features of the game character are input into a preset multilayer neural network. The output of the preset multilayer neural network is input into a first branch. The first branch obtains the predicted motion information and the predicted motion direction information of the game character based on the output of the preset multilayer neural network and preset weights. The loss function of the predicted motion information and the first-level label and the predicted motion direction information and the second-level label is calculated. After inputting the motion features of the game character into the preset multilayer neural network, the method further includes: obtaining the first weight of the game character, including: querying the corresponding preset identifier in the weight dictionary according to the identity identifier of the game character; obtaining the weight corresponding to the preset identifier, the weight being used as the first weight of the game character; inputting the output of the preset multilayer neural network into a second branch; the second branch obtaining the predicted motion information and the predicted motion direction information of the game character based on the first weight of the game character and the output of the preset multilayer neural network; If the loss function satisfies the preset convergence condition, training stops, and the AI ​​model is obtained.

2. The method according to claim 1, characterized in that, Before obtaining a training sample of a game character from the game battle, the method further includes: Obtain a first score for the offline data of the game character, wherein the first score is used to represent the quality of the offline data of the game character; Obtain offline data of the game character whose first rating is greater than or equal to a preset rating.

3. The method according to claim 1, characterized in that, Before obtaining a training sample of the game character from the game battle, the method further includes: Obtain the rank information corresponding to the offline data of the game character. The rank information is used to represent the combat level of the offline data of the game character. Merge the offline data of game characters with the same rank information.

4. The method according to claim 1, characterized in that, The step of obtaining a training sample of the game character from game battles includes: The offline data and corresponding tags of all game characters in the game battle are sent to the data loader so that the data loader can batch process the offline data of all game characters. Obtain the offline data and corresponding tags of all game characters after batch processing by the data loader.

5. The method according to any one of claims 1-4, characterized in that, The feature extraction of the training samples includes: The preset action feature keywords are searched in the offline data of the game character to obtain the action features of the game character.

6. A motion control method, characterized in that, include: Acquire real-time battle data of the game's AI agents; The real-time battle data of the intelligent agent is input into the AI ​​model for feature extraction to obtain action features. The AI ​​model is trained by the AI ​​model training method described in any one of claims 1-5. The AI ​​model outputs the game character's action information and direction information based on the action features, so that the game character executes the action information according to the direction information.

7. A server, characterized in that, The server includes a processor, a memory, and a computer program stored in the memory and executed by the processor. The memory stores a policy model. When the computer program is executed by the processor, it implements the AI ​​model training method as described in any one of claims 1 to 5; or, implements the action control method as described in claim 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the AI ​​model training method as described in any one of claims 1 to 5; or, to implement the motion control method as described in claim 6.

Images