A method, apparatus, device, and storage medium for controlling game artificial intelligence.

By simulating game matches using reinforcement learning strategy models, the win rate and characteristic strength of new game characters on the official server are estimated, solving the problems of long development cycles and prediction bias for new game characters and achieving efficient game character development.

CN116943220BActive Publication Date: 2026-11-10SHENZHEN TENCENT NETWORK INFORMATION TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202310419793.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-04-14
Publication Date
2026-11-10
Estimated Expiration
2043-04-14

Smart Images

  • Figure CN116943220B_ABST
    Figure CN116943220B_ABST
Patent Text Reader

Abstract

This application relates to the field of game technology, and in particular to a game artificial intelligence control method, device, equipment, and storage medium. The method involves: obtaining multiple game character combinations based on selected original game characters and new game characters; executing simulated game matches based on the multiple game character combinations to obtain corresponding game battle data; and using reinforcement learning to train a strategy model to simulate human players controlling new game characters to complete simulated matches, thereby predicting the actual win rate and characteristic strength of the new game characters after they are launched on the official server, thus improving game development efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of game technology, and in particular to a game artificial intelligence control method, device, equipment and storage medium. Background Technology

[0002] With the rapid development of internet technology, online games are becoming increasingly popular. To enhance the richness and playability of games, game designers typically create a large number of selectable game characters and equipment, giving players more freedom of choice. However, whether in multiplayer or single-player games, when players use their favorite character or choose a piece of equipment that is significantly weaker than other options, they will have a poor and unfair experience. This leads to the character or equipment being gradually abandoned by players, reducing the game's richness and playability.

[0003] Under relevant technologies, in order to ensure that there are no options in the game that are significantly stronger or weaker than other similar designs, taking a new game character as an example, the new game character is usually first released on the test server for players to try out. The game designers then conduct statistical analysis on the game data of the players on the test server to estimate the overall strength of the new game character in the official server.

[0004] However, the test server has fewer players, and their mentality differs significantly from that of the official server. For example, players on the test server tend to be casual and lack competitive spirit. Therefore, using this method to predict the overall strength of new game characters will result in significant errors. Furthermore, each multiplayer game match typically lasts more than ten minutes, requiring a considerable amount of time to collect match data using this method, thus extending the development cycle of new game characters. Summary of the Invention

[0005] This application provides a game artificial intelligence control method and related device to improve the development efficiency of new game characters.

[0006] In a first aspect, embodiments of this application provide a game artificial intelligence control method, the method comprising:

[0007] Based on the selected original game characters and new game characters, multiple combinations of game characters can be obtained;

[0008] Based on multiple game character combinations, simulated game matches are executed separately to obtain corresponding game battle data. When executing a simulated game match based on a single game character combination, the following operations are performed:

[0009] At each preset number of game frames, based on the game state information represented by the current game frame, corresponding initial action commands are obtained for each game character in at least one subsequent game frame.

[0010] Preset and adjust each initial action command to obtain the corresponding target action command, and use each target action command to control the corresponding game character to complete the combat action.

[0011] Secondly, embodiments of this application also provide a game artificial intelligence control device, the device comprising:

[0012] The grouping module is used to obtain multiple combinations of game characters based on selected original and new game characters;

[0013] The processing module is used to execute simulated game matches based on multiple combinations of game characters, and obtain corresponding game battle data. When executing a simulated game match based on a single combination of game characters, the following operations are performed:

[0014] At each preset number of game frames, based on the game state information represented by the current game frame, corresponding initial action commands are obtained for each game character in at least one subsequent game frame.

[0015] Preset and adjust each initial action command to obtain the corresponding target action command, and use each target action command to control the corresponding game character to complete the combat action.

[0016] In one possible implementation, when the processing module obtains the corresponding initial action commands for each game character in at least one subsequent game frame based on the game state information represented by the current game frame, the processing module is used to:

[0017] Based on game state information, extract the background space features of the current game frame, the combat power features of each game character, and the game state features of the current game frame.

[0018] The background space features and game state features are respectively fused with the combat power features of each game character to obtain multiple target fusion features;

[0019] Based on the fusion features of multiple targets, corresponding initial action commands are obtained for each game character in at least one subsequent game frame.

[0020] In one possible implementation, when pre-adjusting each initial action command to obtain the corresponding target action command, the processing module is used to:

[0021] Anthropomorphic parameter sets are obtained based on the historical combat data of each original game character. These anthropomorphic parameter sets are used to control the delay state of combat actions.

[0022] Each initial action command is adjusted based on a human-like parameter set to obtain the corresponding target action command.

[0023] In one possible implementation, when obtaining the anthropomorphic parameter set based on the historical combat data of each original game character, the processing module is used for:

[0024] Based on the original game characters, multiple sample game character combinations were obtained, and sample game simulations were executed on each of the multiple sample game character combinations to obtain corresponding sample game battle data.

[0025] Based on the obtained game battle data from each sample, the strength test results for each original game character were obtained.

[0026] Based on the historical combat data of each original game character, the overall combat strength of each original game character is obtained;

[0027] Based on the strength test results of each original game character and their overall combat strength, a set of anthropomorphic parameters was obtained.

[0028] In one possible implementation, when executing simulated matches of sample games based on multiple combinations of sample game characters to obtain corresponding sample game battle data, the processing module is used to:

[0029] When performing a sample game simulation based on a sample game character combination, perform the following operations:

[0030] For each preset number of sample game frames, based on the game state information represented by the current sample game frame, for each game character in at least one subsequent sample game frame, multiple candidate sample commands are obtained, as well as the benefit evaluation values ​​corresponding to the multiple candidate sample commands. The benefit evaluation values ​​are used to measure the benefits brought to the sample game simulation game after the candidate sample commands are executed.

[0031] Based on the revenue evaluation values ​​corresponding to multiple candidate sample instructions, the weight matrix corresponding to the game state information represented by the current sample game frame is determined, and based on the weight matrix, the execution probability corresponding to each of the multiple candidate sample instructions is determined.

[0032] Based on the execution probabilities of multiple candidate sample instructions, target sample instructions that meet the preset probability requirements are selected for each game character, and each target sample instruction is used to control the corresponding game character to complete combat actions.

[0033] In one possible implementation, after obtaining the strength test results for each original game character based on the acquired sample game battle data, the processing module is further used for:

[0034] Based on the original and new game characters, multiple new sample game character combinations were obtained. Based on these new sample game character combinations, new sample game simulation matches were executed to obtain corresponding new sample game battle data.

[0035] Based on the newly obtained sample game battle data, the strength test change rate of the new game characters was obtained;

[0036] When the rate of change of the intensity test meets the preset convergence condition, it is determined that the control environment for artificial intelligence control of the new game character has reached a stable state.

[0037] In one possible implementation, the revenue assessment value includes at least one of the following:

[0038] The first sub-benefit evaluation value characterizing the experience and resource growth brought to the current game character by the candidate sample instruction;

[0039] The second sub-benefit evaluation value represents the kill and assist rewards that the candidate sample instruction brings to the current game character.

[0040] The third sub-profit evaluation value that characterizes the candidate sample instruction that results in the win or loss of the current sample simulation game.

[0041] In one possible implementation, before obtaining multiple combinations of game characters from a pool of selected original and new game characters, the processing module is further used to:

[0042] Obtain the target version information of the game version to be launched, and update the execution environment of the game simulation based on the differences between the target version information and the historical version information corresponding to the running historical version.

[0043] In one possible implementation, based on multiple combinations of game characters, after executing simulated game matches and obtaining the corresponding game battle data, the processing module is further used for:

[0044] Based on the obtained game battle data, the strength test results corresponding to the new game character are obtained. The strength test results corresponding to the new game character include the classification test results corresponding to multiple test indicators, and corresponding strength threshold ranges are preset for multiple test indicators.

[0045] In one possible implementation, after obtaining the strength test results for the new game character based on the acquired game battle data, the processing module is further used for:

[0046] The results of each category of the strength test are compared with their respective strength threshold ranges to obtain the comparison results.

[0047] If the comparison results indicate that the test results of each category do not exceed their respective intensity threshold range, then the new game character is determined to meet the preset balance requirements.

[0048] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any of the first aspects.

[0049] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described in the first aspect.

[0050] Fifthly, embodiments of this application provide a computer program product that, when invoked by a computer, causes the computer to execute the method described in the first aspect.

[0051] The beneficial effects of this application are as follows:

[0052] This application proposes a game AI control method, device, equipment, and storage medium. The server obtains multiple game character combinations based on selected original and new game characters. Based on these combinations, it executes simulated game matches to obtain corresponding game battle data. By using reinforcement learning to train a strategy model to simulate human players controlling new game characters in simulated matches, and obtaining battle data through the output of the strategy model, the actual win rate and characteristic strength of the new game character after its official release on the live server can be predicted. This significantly shortens the development cycle of new game characters and improves game development efficiency.

[0053] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0054] Figure 1 This is a schematic diagram illustrating possible application scenarios in the embodiments of this application;

[0055] Figure 2 This is a schematic diagram illustrating the process of training the initial policy model and obtaining the anthropomorphic parameter set in an embodiment of this application;

[0056] Figure 3 This is a schematic diagram illustrating the process of sample game character combinations performing a sample game simulation in an embodiment of this application;

[0057] Figure 4This is a map thumbnail of a MOBA game provided in the embodiments of this application;

[0058] Figure 5 This is a schematic diagram of the data processing process of the strategy model in the embodiments of this application;

[0059] Figure 6A This is a schematic diagram of the first scene of a sample game simulation provided in the embodiments of this application;

[0060] Figure 6B This is a schematic diagram of the second scene of the sample game simulation provided in the embodiments of this application;

[0061] Figure 6C This is a schematic diagram of a third scenario of a simulated game match provided in the embodiments of this application;

[0062] Figure 7 This is a flowchart illustrating the training process of the target policy model in an embodiment of this application.

[0063] Figure 8 This is a detailed flowchart illustrating the artificial intelligence control of the new game character in the embodiments of this application;

[0064] Figure 9 This is a schematic diagram of a hero combination provided in an embodiment of this application;

[0065] Figure 10 This is a schematic diagram illustrating the process of game character combinations performing a simulated game match in an embodiment of this application;

[0066] Figure 11 This is a schematic diagram of the first network output of the target strategy model provided in the embodiments of this application;

[0067] Figure 12 This is a schematic diagram of game battle data provided in an embodiment of this application;

[0068] Figure 13 This is an example diagram illustrating the process of testing the balance of new game characters in this application embodiment;

[0069] Figure 14 A logical diagram of a game artificial intelligence control method in an embodiment of this application;

[0070] Figure 15 This is a schematic diagram of the structure of the game artificial intelligence control device provided in the embodiments of this application;

[0071] Figure 16 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application;

[0072] Figure 17This is a schematic diagram of the hardware structure of another computer device in an embodiment of this application. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are only some embodiments of the technical solutions of this application, and not all embodiments. Based on the embodiments recorded in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the technical solutions of this application.

[0074] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0075] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0076] Reinforcement Learning (RL) is the third fundamental machine learning method, besides supervised and unsupervised learning. It focuses on finding a balance between exploration (of the unknown) and utilization (of existing knowledge). Inspired by behaviorism in psychology, it explores how organisms, under the influence of rewards or punishments from their environment, gradually develop expectations of stimuli and produce habitual behaviors that maximize benefits. Research areas related to this method include autonomous driving and game AI.

[0077] Multiplayer Online Battle Arena (MOBA): Also known as multiplayer online battle arena games, or action real-time strategy (ARTS) games, these games originate from real-time strategy (RTS) games and are a subgenre of massively multiplayer online (MMO) games. Players are typically divided into two teams, with each player usually controlling only one character from one team. Victory is achieved by destroying the opposing team's base structures. Representative games include: Honor of Kings, Dota 2, and League of Legends.

[0078] Convolutional Neural Networks (CNNs) are a type of feedforward neural network that includes convolutional computation and has a deep structure. They are one of the representative algorithms of deep learning. CNNs have representation learning capabilities and can perform translation-invariant classification of input information according to their hierarchical structure.

[0079] Long Short-Term Memory (LSTM): This is a type of recurrent neural network (RNN) with long-term memory capabilities. Its network structure consists of one or more units with forgetting and remembering functions. Key components include the Forget Gate, Input Gate, and Output Gate, which are responsible for deciding whether the current input is accepted, whether it is stored in memory, and whether the input in memory is output in the current context. Because of its memory function, LSTM is frequently used in scenarios and situations with time-series data.

[0080] The design concept of the embodiments of this application is briefly introduced below:

[0081] Under the relevant technology, in order to ensure that there are no options in the game that are significantly stronger or weaker than other similar designs, taking a new game character as an example, it is necessary to conduct balance tests on the new game character to be launched before the game version is updated.

[0082] Currently, commonly used methods for balance testing of new game characters include:

[0083] Method 1: Expert Knowledge Prediction

[0084] The design of new game characters is completed by the game designers. Based on the game designers' understanding of the game mechanics, combined with the evaluation of existing game character mechanics and values, the strength of the new game characters is predicted by the expert experience of the planners.

[0085] When using Method 1, the results of expert knowledge predictions heavily rely on the game designers' subjective understanding and judgment of the game. Furthermore, since the mechanics and numerical attributes of new game characters are entirely new, the results of predictions based on experiential knowledge will deviate significantly from the actual strength of the new game characters as determined by player testing after the game's release.

[0086] Method 2: Regression Model Prediction

[0087] The regression model is based on external static data indicators, such as kills and economy, to construct ability models for different types of game characters. Based on the characteristics of new game characters, the strength of new game characters is predicted using ability models of similar game characters.

[0088] When using Method 2, there is a problem of missing data for new game characters. In addition, the release of new game characters is accompanied by many adjustments to the game. Historical data cannot accurately estimate the new design changes, resulting in a large error in the prediction of the strength of new game characters.

[0089] Method 3: Experience Server Testing

[0090] New game characters are first released on the test server for players to try out. Game designers then use the statistical data from players on the test server to estimate the strength of the new game characters on the official server.

[0091] When using Method 3, the game settings on the test server differ from those on the official server. For example, the map design, equipment, and character versions on the test server are different, making it insufficient to reflect the game environment of the official server. Furthermore, the test server has fewer players, and their mentality differs significantly from that on the official server; for instance, players on the test server may be more casual and lack competitive spirit. Therefore, using this method to predict the overall strength of new game characters has a significant bias. Additionally, each multiplayer game match typically lasts more than ten minutes, requiring a considerable amount of time to collect match data using this method, resulting in a prolonged development and testing cycle for new game characters.

[0092] In view of this, this application proposes a game artificial intelligence control method, device, equipment, and storage medium. The server obtains multiple game character combinations based on selected original and new game characters. Based on these combinations, it executes simulated game matches to obtain corresponding game battle data. When executing a simulated game match based on a single game character combination, the server performs the following operations using a pre-trained strategy model: at preset intervals of a certain number of game frames, based on the game state information represented by the current game frame, it obtains corresponding initial action commands for each game character in at least one subsequent game frame. It then performs preset adjustments on these initial action commands to obtain corresponding target action commands, and uses these target action commands to control the corresponding game characters to complete battle actions.

[0093] In this way, by using reinforcement learning to train a strategy model, we can simulate human players controlling new game characters to complete simulated matches. By obtaining the battle data through the output of the strategy model, we can predict the actual win rate and characteristic strength of the new game character after it is launched on the official server. This reduces the risk of subjective bias in expert knowledge prediction and solves the defect of regression model prediction relying on static historical data. At the same time, compared with the test period of at least one month on the experience server, this solution can obtain test results within two weeks, which greatly shortens the testing cycle of new game characters and thus improves game development efficiency.

[0094] In the embodiments of this application, the initial policy model and the target policy model involve artificial intelligence (AI) and machine learning technologies, and are designed based on speech technology, natural language processing technology and machine learning (ML) in artificial intelligence.

[0095] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence.

[0096] Artificial intelligence (AI) studies the design principles and implementation methods of various intelligent machines, enabling them to perceive, reason, and make decisions. AI technology mainly includes computer vision, natural language processing, and machine learning / deep learning. With the research and advancement of AI technology, it is being researched and applied in multiple fields, such as smart homes, intelligent customer service, virtual assistants, smart speakers, intelligent marketing, autonomous driving, robotics, and smart healthcare. It is believed that with further technological development, AI will be applied in even more fields and play an increasingly important role.

[0097] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Compared to data mining, which focuses on finding patterns in large datasets, machine learning emphasizes algorithm design, enabling computers to automatically "learn" patterns from data and use these patterns to predict unknown data.

[0098] Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning. Reinforcement learning (RL), also known as reward learning, evaluation learning, or enhancement learning, is one of the paradigms and methodologies of machine learning. It is used to describe and solve problems where an agent learns strategies to maximize rewards or achieve specific goals during its interaction with the environment.

[0099] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0100] See Figure 1 The diagram shown is a possible application scenario illustration in the embodiments of this application.

[0101] This application scenario includes terminal device 110 (including terminal device 1101, terminal device 1102... terminal device 110n) and server 120. Terminal device 110 and server 120 can communicate with each other through a communication network.

[0102] In one alternative implementation, the communication network can be a wired network or a wireless network. Therefore, the terminal device 110 and the server 120 can be connected directly or indirectly via wired or wireless communication. For example, the terminal device 110 can be indirectly connected to the server 120 via a wireless access point, or the terminal device 110 can be directly connected to the server 120 via the Internet; this application does not impose any limitations on this.

[0103] In this application embodiment, the terminal device 110 includes, but is not limited to, mobile phones, tablets, laptops, desktop computers, e-book readers, smart voice interaction devices, smart home appliances, vehicle terminals, and other devices; various clients can be installed on the terminal device, which can be applications that support video preview, video playback, and other functions (such as browsers, game software, etc.), or web pages, mini programs, etc.

[0104] Server 120 is a backend server corresponding to the client installed in terminal device 110. Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0105] It should be noted that the game artificial intelligence control method in this application embodiment can be executed by a computing device, which can be a server 120 or a terminal device 110. That is, the method can be executed by the server 120 or the terminal device 110 alone, or by the server 120 and the terminal device 110 together.

[0106] It should be noted that the following text mainly uses the example of the server running alone, and no specific limitations are made here.

[0107] It should be noted that, Figure 1 The examples shown are merely illustrative; in reality, the number of terminal devices 110 and servers 120 is not limited and is not specifically limited in this embodiment.

[0108] In this embodiment of the application, before the server performs artificial intelligence control on the new game character, it first trains the strategy model to be trained based on the selected original game characters to obtain an initial strategy model, and obtains an anthropomorphic parameter set through the initial strategy model.

[0109] See Figure 2 As shown, this is a flowchart illustrating the process of training the initial policy model and obtaining the anthropomorphic parameter set in an embodiment of this application. The following is a detailed explanation in conjunction with the attached diagram. Figure 2 The specific operations to be performed will be explained in detail:

[0110] Step 201: Based on each original game character, obtain multiple sample game character combinations, and based on the multiple sample game character combinations, execute sample game simulation matches respectively to obtain corresponding sample game battle data.

[0111] Specifically, in this embodiment, the strategy model is a neural network model based on reinforcement learning, which is used to control each game character to replace human players in battle. Before training the initial strategy model, it is necessary to obtain multiple sample game character combinations based on the game character pool composed of each original game character according to the game rules. Then, the strategy model to be trained controls each sample game character combination to execute a large number of sample game simulations, thereby training and obtaining the initial strategy model.

[0112] For example, in a common MOBA game, players are usually divided into two teams of five. The two teams compete against each other on a scattered game map. The ultimate goal is to destroy the opponent's main base to win. Therefore, the server randomly divides multiple 5v5 sample game character combinations based on the game character pool composed of the original game characters and according to reasonable lineup rules. The strategy model controls each game character to complete the sample simulation game.

[0113] Among them, see Figure 3 As shown, it is a schematic diagram of the process of sample game character combinations performing sample game simulation matches in an embodiment of this application. The following is in conjunction with the attached diagram. Figure 3 The specific operations to be performed will be explained in detail:

[0114] When performing a sample game simulation based on a sample game character combination, the server performs the following operations:

[0115] Step 2011: At each preset number of sample game frames, based on the game state information represented by the current sample game frame, for each game character in at least one subsequent sample game frame, obtain the corresponding multiple candidate sample commands and the corresponding benefit evaluation values ​​of the multiple candidate sample commands.

[0116] Among them, the benefit assessment value is used to measure the benefit brought to the sample game simulation after the execution of the candidate sample instruction.

[0117] Specifically, the input of the strategy model is the game state information contained in each frame of the game, and the output is the action instructions to control each game character to complete the combat actions. Every 3 game frames, the strategy model extracts the background space features of the current game frame, the combat power features of each game character, and the game state features of the current game frame from the game state information of the current game frame. The current game frame is called a decision frame, and the remaining frames are non-decision frames.

[0118] For example, see Figure 4 As shown, this is a map thumbnail of a MOBA game provided in this application embodiment. The map contains multiple game characters, game resources such as monsters and minions, and multiple defensive tower buildings. Taking a game character A in the current game simulation as the main perspective, the strategy model captures the image of the area where game character A is located in the current game frame and extracts the background space features. In this application embodiment, the background space features are tensor features of image type, with multiple channels. The matrix of each channel represents a feature of one dimension, namely the damage channel, control channel, map channel, and bush channel. The size of each channel is a 17×17 matrix.

[0119] Furthermore, the strategy model extracts the combat characteristics of each game character, specifically including the current health, attack power, position, skills, equipment, and other attribute characteristics of each game character.

[0120] Optionally, in addition to the combat power characteristics of each game character, the strategy model can also extract the unit characteristics of each game resource and building within the current game frame, such as the health, attack power, and position of all minions and monsters, as well as the position of defensive towers relative to each game character.

[0121] In addition, the strategy model extracts the game state features of the current game frame, including the time stage of the game, the resource ratio of both sides, the number of surviving game characters of both sides, and the number of remaining defensive towers of both sides.

[0122] After obtaining the aforementioned high-dimensional features, the strategy model performs feature processing, including, as described in [reference needed]. Figure 5As shown, this is a schematic diagram of the data processing process of the strategy model in an embodiment of this application. The background spatial features of the image type are processed by two layers of convolutional neural networks (e.g., Figure 5 As shown in the CNN), the vector-type combat power features, unit features, and game state features are respectively processed by different multilayer perceptron networks (such as...). Figure 5 The MLP1 and MLP2 shown are used for feature processing. All processed features are then passed through a feature fusion layer (such as MLP1 and MLP2). Figure 5 As shown in the concatenate diagram, feature fusion is performed to obtain the target fused features.

[0123] For example, the target fusion feature 'a' corresponding to game character A is obtained by fusing the background space features of the image area obtained from the main perspective of game character A, the combat power features of game character A, the other unit features of the current game frame, and the game state features of the current game frame. If the image area from the main perspective of game character A also contains the enemy game character B, then the target fusion feature 'a' also includes the combat power features of game character B.

[0124] After the policy model continuously obtains the target fusion features of 16 decision frames, the target fusion features of the 16 frames are input into the Long Short-Term Memory network (e.g., Figure 5 The LSTM shown is used to capture the temporal information of the game. Finally, the output of the Long Short-Term Memory network is transmitted to the Full Connection (FC) network to parse and obtain multiple candidate sample instructions corresponding to each game character, as well as the reward evaluation value corresponding to the multiple candidate sample instructions.

[0125] Specifically, in the embodiments of this application, the benefit evaluation value corresponding to each candidate sample instruction includes at least one of the following:

[0126] (1) The first sub-benefit evaluation value that represents the experience and resource growth brought to the current game character by the candidate sample instruction.

[0127] For example, see Figure 6A As shown, it is a schematic diagram of the first scene of the sample game simulation provided in the embodiment of this application. If the candidate sample instruction 1 corresponding to the game character X is to release a skill, and there is a group of minions within the attack range of the game character A, then the first sub-benefit evaluation value corresponding to the candidate sample instruction 1 is: the candidate sample instruction 1 brings m points of experience growth and n points of resource growth to the game character X.

[0128] (2) Characterize the second sub-benefit evaluation value that the candidate sample instruction brings kills and assists to the current game character.

[0129] For example, see Figure 6BAs shown, it is a schematic diagram of the second scenario of the sample game simulation provided in the embodiment of this application. If the candidate sample instruction 2 corresponding to the game character X is to release the third skill, and there is an enemy game character Y with low health within the attack range of the game character A, then the second sub-benefit evaluation value corresponding to the candidate sample instruction 2 is: the candidate sample instruction 2 is an increase in the kill value brought by the game character X.

[0130] (3) The third sub-gain evaluation value that characterizes the outcome of the current sample simulation game caused by the candidate sample instruction.

[0131] For example, see Figure 6C As shown, it is a schematic diagram of the third scenario of the sample game simulation provided in the embodiment of this application. If the candidate sample instruction 3 corresponding to the game character X is a normal attack, and the attack range of the game character X includes the enemy's low-health main base, then the third sub-benefit evaluation value corresponding to the candidate sample instruction 3 is: the candidate sample instruction 3 leads to the increase of the benefit of winning the current sample simulation game. The specific benefit evaluation value is automatically allocated by the model settings.

[0132] It should be noted that the above examples are only for the purpose of demonstrating the different sub-benefit evaluation values ​​corresponding to various candidate sample instructions in different scenarios. Therefore, candidate sample instruction 1, candidate sample instruction 2 and candidate sample instruction 3 do not appear in the same scenario. In the actual game simulation during the training process, each candidate sample instruction should be generated for the same game character in the same scenario.

[0133] Step 2012: Based on the revenue evaluation values ​​corresponding to multiple candidate sample instructions, determine the weight matrix corresponding to the game state information represented by the current sample game frame, and based on the weight matrix, determine the execution probability corresponding to each of the multiple candidate sample instructions.

[0134] Specifically, the strategy model updates the benefit evaluation value of each candidate sample instruction based on the benefit evaluation value corresponding to each candidate sample instruction using the gradient ascent algorithm. At the same time, it determines the weight matrix corresponding to the current game state information, and selects the optimal weight matrix based on the historical weight matrices, including this weight matrix. Then, based on the optimal weight matrix and combined with the current game state information, it determines the execution probability of each candidate sample instruction.

[0135] For example, as mentioned above, if the attack range of game character A simultaneously includes a minion wave, a low-health enemy game character Y, and a low-health enemy main base, and the candidate sample commands of game character A include candidate sample command 1, candidate sample command 2, and candidate sample command 3, then the strategy model is based on the benefit evaluation value corresponding to each candidate sample command. Among them, candidate sample command 2 can bring game character X not only a kill value increase, but also an experience increase and a resource increase. Therefore, its benefit evaluation value is the sum of the first sub-evaluation value and the second sub-evaluation value.

[0136] Based on the benefit evaluation value corresponding to each candidate sample instruction, a gradient ascent algorithm (such as the dual near-end gradient optimization algorithm) is used to update the benefit evaluation value of each candidate sample instruction. At the same time, the weight matrix corresponding to the current game state information is determined, and the weight matrix is ​​added to the historical weight matrix pool. The optimal weight matrix is ​​selected from the pool, and the execution probability of each candidate sample instruction is determined by combining the current game state information.

[0137] Step 2013: Based on the execution probabilities of multiple candidate sample instructions, select target sample instructions that meet the preset probability requirements for each game character, and use each target sample instruction to control the corresponding game character to complete the combat action.

[0138] Specifically, the strategy model selects the target sample instruction with the highest execution probability from multiple candidate sample instructions corresponding to each game character based on the execution probability of each candidate sample instruction, and transmits each target sample instruction to the game engine responsible for game advancement, so that the game engine controls each game character to complete the corresponding combat actions.

[0139] In the initial stage of training the policy model based on reinforcement learning, the reward evaluation value corresponding to each candidate sample instruction is set to 0. The policy model continuously tries various candidate sample instructions and continuously obtains the reward value after executing the corresponding decision. After processing 16 decision frames, the reward evaluation value corresponding to each candidate sample instruction is updated once through the gradient ascent algorithm. Finally, after training for multiple rounds, the evaluation value corresponding to each candidate sample instruction will tend to a stable value, and the optimal weight matrix will also remain unchanged.

[0140] Step 202: Based on the obtained game battle data of each sample, obtain the strength test results of each original game character.

[0141] Furthermore, in this embodiment of the application, the server controls multiple sample game character combinations through a strategy model, executes sample game simulations respectively, obtains corresponding sample game battle data, trains and obtains a stable initial strategy model, and then, based on the battle data of each sample game, statistically obtains the strength test results corresponding to each original game character, including at least the predicted average win rate of each original game character.

[0142] For example, the server retrieves all sample game battle data containing game character A, calculates the win and loss of each sample simulated game, and obtains a predicted average win rate of 56.7% for game character A.

[0143] In addition to predicting the average win rate, the strength test results may also include the appearance rate and ban rate of the original game characters, etc., which are not restricted in this application.

[0144] Step 203: Based on the historical combat data of each original game character, obtain the overall combat strength of each original game character.

[0145] Specifically, each original game character has already been tested by a large number of real players on the official server, resulting in a large amount of practical data. Based on the historical practical data of each original game character obtained by real players in the one to two weeks before the release of the game version, the overall practical strength of each original game character can be statistically determined, including at least the historical average win rate of each original game character.

[0146] For example, the server retrieves all historical match data for game character A from the official server, calculates the wins and losses of each match, and obtains a historical average win rate of 52.72% for game character A.

[0147] Step 204: Based on the strength test results of each original game character and the overall combat strength of each original game character, obtain the anthropomorphic parameter set.

[0148] Specifically, the server adjusts the anthropomorphic parameter set of the initial strategy model based on the predicted average win rate and the historical average win rate of each original game character, through a preset adjustment logic. Then, it re-obtains the strength test results of each original game character through the adjusted initial strategy model until the error between the predicted average win rate of each original game character predicted by the initial strategy model and the historical average win rate of real players is within a preset error range, thus obtaining the final anthropomorphic parameter set.

[0149] Among them, the anthropomorphic parameter set includes at least the action delay parameter, which is used to control the delay state of combat actions, so that when the model controls each game character to complete combat actions, it is more in line with the reaction speed of real players.

[0150] For example, as shown in Table 1, it is a comparison table of win rate difference ranges and anthropomorphic parameter adjustment amounts provided in the embodiments of this application:

[0151] Table 1

[0152]

[0153] Table 1 records the anthropomorphic parameter adjustment strategies corresponding to the difference range between the predicted average win rate and the historical average win rate. The difference between the predicted average win rate (56.75%) and the historical average win rate (52.72%) of game character A is 4.03%. Therefore, based on this difference, the corresponding anthropomorphic parameter adjustment amounts within the win rate difference range are: increase the action delay parameter by 5%, increase the skill release frequency adjustment by 0.5%, and so on...

[0154] It should be noted that the anthropomorphic parameter adjustment strategies recorded in Table 1 above are only for illustrating this application and are not intended to limit this application.

[0155] Furthermore, due to the limited generalization ability of reinforcement learning models, directly using the initial policy model to test the strength of new game characters will cause the referenced external environment to be changed when the initial policy model controls each original game character. That is, the initial policy model cannot recognize the attribute characteristics of the new game character and cannot output the correct prediction instructions for each game character, thus failing to obtain more stable strength test results.

[0156] Therefore, after the server obtains the initial strategy model and anthropomorphic parameter set, it still needs to add new game characters to the game character pool, retrain the initial strategy model, and obtain the target strategy model. (See [link to relevant documentation]). Figure 7 As shown, it is a flowchart illustrating the training of the target policy model in an embodiment of this application. The following is a detailed explanation in conjunction with the attached diagram. Figure 7 The specific execution steps are explained in detail below:

[0157] Step 701: Based on the original game characters and the new game characters, obtain multiple new sample game character combinations, and based on the multiple new sample game character combinations, execute new sample game simulation matches to obtain corresponding new sample game battle data.

[0158] Specifically, the server inputs configuration information such as the skill effects, skill cooldown time, and equipment order of the new game characters into the initial strategy model. Based on the game character pool composed of the original game characters and the new game characters, it obtains multiple new sample game character combinations. Then, it controls each new sample game character combination through the initial strategy model to execute a large number of new sample game simulation matches, thereby training and obtaining the target strategy model.

[0159] Assume the new game character's third skill has the following effect: it generates a vortex to absorb enemy damage and then counterattacks the absorbed damage back to the enemy unit after the vortex ends. During this time, the character is in a state of invulnerability and immune to control effects. The cooldown is 45 seconds. After the server inputs the above information into the initial strategy model, when a sample game simulation is performed based on the new game character and each original game character, each original game character can obtain environmental information about the new game character. This allows the initial strategy model to output the correct target sample command for each game character. For example, if in the current game frame, the new game character uses its third skill to attack enemy game character B, since the new game character's third skill has the effect of absorbing enemy damage, the target sample command output by the initial strategy model for game character B is very likely to be not to use the skill.

[0160] Step 702: Based on the newly obtained sample game battle data, obtain the strength test change rate of the new game character.

[0161] After obtaining a fixed number of sample game battles, the strength test results of the new game character are statistically analyzed, specifically including at least the predicted average win rate of the new game character, and the strength test change rate of the new game character is recorded, specifically including at least the change rate of the predicted win rate of the new game character.

[0162] For example, after obtaining 300,000 sample game battles, the predicted average win rate of the new game character is calculated based on the wins and losses of each sample game simulation match that includes the new game character. After obtaining 15 million sample game battles, the change curve of the predicted average win rate of the new game character is obtained.

[0163] It should be noted that during the training of the target strategy model, the stable win rate of each original game character obtained in the initial strategy model training phase may be affected. Therefore, after obtaining the target strategy model, the initial strategy model training and anthropomorphic parameter adjustment steps described above will be repeated until the final anthropomorphic parameter set and target strategy model are obtained.

[0164] The difference between training the target policy model and training the initial policy model is that when training the target policy model, new game characters will participate in the division of sample game character combinations. The rest of the steps are the same, and no further examples will be given here.

[0165] Step 703: When the rate of change of the intensity test meets the preset convergence condition, it is determined that the control environment for artificial intelligence control of the new game character has reached a stable state.

[0166] Specifically, when the rate of change in the strength test of a new game character reaches a stable state, for example, when the curve of the predicted average win rate of the new game character reaches a stable state, it can be considered that the initial strategy model has been trained to obtain a stable target strategy model. Then, the target strategy model can be used to control each game character, including the new game character, to perform game simulation matches, thereby conducting balance tests on the new game character.

[0167] See Figure 8 As shown, this is a detailed flowchart of the balance test for a new game character in an embodiment of this application. The following is a description of the process in conjunction with the attached diagram. Figure 8 Taking the 5v5 game scenario of Honor of Kings as an example, the specific execution steps will be explained in detail:

[0168] Step 801: Based on the selected original game characters and new game characters, obtain multiple combinations of game characters.

[0169] Specifically, before executing step 801, the server obtains the target version information of the version to be launched in advance, and updates the execution environment of the game simulation based on the difference between the target version information and the historical version information corresponding to the historical version in operation.

[0170] For example, the differences between the target version information and the historical version information of the version to be launched may include other updates such as art design, equipment adjustments and map optimization, in addition to new game characters. Therefore, before testing a new game character, it is necessary to update the game engine that controls the game based on these differences to ensure that the version information of the new game character during testing is exactly the same as the version information after the new game character is launched on the official server, thereby ensuring that the balance test for the new game character is more accurate.

[0171] Furthermore, apart from a few original game characters that are undergoing rework or adjustment in the official server, the server will obtain multiple game character combinations based on the game character pool composed of original and new game characters, according to the lineup matching rules.

[0172] For example, suppose there are 99 original heroes (i.e., original game characters) in Honor of Kings. Add the new hero to be tested (i.e., new game character) to them to form a hero pool of 100. Divide the heroes into multiple 5v5 hero combinations according to the team composition rules. The types of the 5 heroes on both sides are: assassin, mage, warrior, marksman and roamer.

[0173] See Figure 9 As shown, this is a schematic diagram of a hero combination provided in an embodiment of this application, wherein the hero Zhao Huaizhen is a new hero to be tested, and his hero type is warrior.

[0174] It should be noted that when multiple hero combinations are obtained, the server determines the appearance rate of the new hero based on the average appearance rate of the original heroes of each warrior type in the official server. In other words, the new hero to be tested does not need to appear in every hero combination.

[0175] Step 802: Based on multiple game character combinations, execute game simulation matches respectively to obtain corresponding game battle data.

[0176] In each simulated game match, the server controls the game characters to perform combat actions using a trained target strategy model. (See also...) Figure 10 As shown, it is a schematic diagram of the process of game character combination performing game simulation in an embodiment of this application. The following is in conjunction with the attached diagram. Figure 10 The specific operations to be performed will be explained in detail:

[0177] When performing a simulated game match based on a combination of game characters, the server performs the following operations based on the target strategy model:

[0178] Step 8021: At each preset number of game frames, based on the game state information represented by the current game frame, obtain the corresponding initial action instructions for each game character in at least one subsequent game frame.

[0179] Specifically, in this embodiment of the application, every 3 game frames, the target strategy model extracts the background space features of the current game frame, the combat power features of each game character, and the game state features of the current game frame based on the game state information of the current game frame.

[0180] Similar to the policy model training phase, the background space features are image-type tensor features with multiple channels. Each channel's matrix represents a feature in one dimension, namely the damage channel, control channel, map channel, and grass channel. Each channel is a 17×17 matrix.

[0181] Each game character's unique combat characteristics include their current health, attack power, position, skills, equipment, and other attributes.

[0182] Optionally, in addition to the combat power characteristics of each game character, the target strategy model can also extract the unit characteristics of each game resource and building within the current game frame, such as the health, attack power, and position of all minions and monsters, as well as the position of defensive towers relative to each game character.

[0183] In addition, the target strategy model extracts the game state features of the current game frame, including the time stage of the game, the resource ratio of both sides, the number of surviving game characters of both sides, and the number of remaining defensive towers of both sides.

[0184] Furthermore, the target strategy model uses the method described above to fuse background space features and game state features with the combat power features of each game character to obtain multiple target fusion features. Based on the multiple target fusion features, corresponding initial action commands are obtained for each game character in at least one subsequent game frame.

[0185] In this embodiment of the application, the initial action instruction specifically includes three parts: target button, target position, and target unit. It is composed of multiple network outputs of the target strategy model. The first output is used to control the movement of the game character or the release of skills, that is, the target button is the movement button or the skill button. The second output is used to control the angle of the movement of the game character, that is, the target position. The third output is used to control the coordinates of the skill release, that is, the target unit.

[0186] For example, see Figure 11 As shown, it is the first network output of the target strategy model provided in this application embodiment. It is a multi-dimensional vector. The first dimension of the vector is 1, which means that the current hero does nothing. The second dimension of the vector is 1, which means that the hero moves. The third dimension of the vector is 1, which means that the hero attacks normally. And so on, predicting the release of the first skill, the second skill, the third skill, the fourth skill, the summoner skill, etc.

[0187] Step 8022: Preset and adjust each initial action command to obtain the corresponding target action command, and use each target action command to control the corresponding game character to complete the combat action.

[0188] Among them, the anthropomorphic parameter set is obtained based on the historical combat data of each original game character and is used to control the delay state of combat actions.

[0189] Specifically, in order to make the target strategy model control each game character more closely resemble the operation of real players, the server will adjust the initial action commands output by the target strategy model based on the pre-obtained anthropomorphic parameter set, and transmit each target action command to the game engine, so that the game engine can control the corresponding game characters to complete the combat actions.

[0190] For example, suppose that in the current game frame, the target strategy model is targeting the hero Zhao Huaizhen, and the initial action command output is to release the first skill to attack the hero Jialuo within range, and the anthropomorphic parameter set contains an action delay parameter of 0.5s, then the adjusted target action command is: release the first skill after a delay of 0.5s.

[0191] Furthermore, based on a combination of game characters, a simulated game match is executed to obtain and save the corresponding game battle data. (See [link / reference]). Figure 12As shown, this is a schematic diagram of game battle data provided in an embodiment of this application. The game battle data for each game includes at least information such as the damage output, damage taken, total resource gains, and participation rate of each game character.

[0192] Because the target strategy model can control multiple game character combinations to simultaneously execute multiple game simulation matches, the server can obtain tens of millions of game battle data within a day, and based on the obtained game battle data, quickly obtain the strength test results corresponding to the new game character.

[0193] Specifically, in this embodiment of the application, the strength test results for the new game character include multiple test indicators, and each test indicator corresponds to a category test result.

[0194] For example, see Table 2, which is a classification table of test indicators provided in the embodiments of this application:

[0195] Table 2

[0196]

[0197] Table 2 provides examples of the classification of some test indicators at different levels, mainly divided into two major levels: in-game and out-of-game. In-game test indicators mainly analyze hero attributes within the game, while out-of-game test indicators mainly analyze statistical information from all games. For example, the first-level ability items analyzed at the in-game level include: development ability, output ability, survivability, cooperation ability, pushing ability, and jungling ability. The first-level ability items analyzed at the out-of-game level include: game win / loss, game classification, and game evaluation.

[0198] Furthermore, since there are preset intensity threshold ranges for multiple test indicators, after the server obtains the intensity test results, it compares the test results of each category included in the intensity test results with their respective intensity threshold ranges to obtain comparison results.

[0199] If the comparison results indicate that the test results of each category do not exceed their respective intensity threshold range, then the new game character is determined to meet the preset balance requirements.

[0200] For example, assuming the server statistics show that the new hero Zhao Huaizhen's in-game output ability is 22.5%, and the preset strength threshold range for this test indicator is 15%-35%, then it means that the new hero Zhao Huaizhen's output ability meets the preset balance requirements.

[0201] Optionally, in addition to comparing the above-mentioned classification test results with the preset strength threshold range, the strength test results of the new game character can also be compared with the overall combat strength of other original game characters of the same type.

[0202] For example, assuming that the strength test results output by the target strategy model indicate that the new hero Zhao Huaizhen has a win rate of 55.37%, and that the average win rate of the original hero Kai, who is also a warrior, is 53.15%, and the win rate of the original hero Arthur is 50.08%, etc., we can roughly estimate the strength ranking of the new hero Zhao Huaizhen.

[0203] Based on the above embodiments, the following uses a specific application scenario to illustrate the process of game artificial intelligence control in this application embodiment. (See attached document.) Figure 13 The diagram shown is an example of the process of using a game artificial intelligence control method to conduct balance testing on a new game character in an embodiment of this application, specifically including:

[0204] Step 1301: Based on each original game character, perform sample game simulations, use reinforcement learning algorithms to train and obtain an initial policy model, and obtain the predicted average win rate of each original game character based on the initial policy model.

[0205] Step 1302: Based on the historical win rate of each original game character and the predicted average win rate of each original game character, obtain the anthropomorphic parameter set.

[0206] Step 1303: Obtain the configuration information of the new game characters, and based on the original game characters and the new game characters, execute a new sample game simulation to retrain the initial strategy model and obtain the target strategy model.

[0207] Step 1304: Update the game version information, and based on the original game characters and the new game characters, use the target strategy model and anthropomorphic parameter set to execute a game simulation match to obtain the strength test results of the new game characters.

[0208] See Figure 14 As shown, this is a logical diagram of a game artificial intelligence control method in an embodiment of this application. First, through the strategy model to be trained, each original game character is controlled to execute simulated game matches with 30 million samples to obtain an initial strategy model. At the same time, the predicted average win rate of each original game character is output based on the initial strategy model. Combined with the known historical average win rate of each original game character, a human-like parameter set is obtained. Second, through the initial strategy model, the original game character and the new game character are controlled to execute simulated game matches with 30 million samples to obtain a target strategy model. Finally, the game version information is updated. Through the target strategy model and combined with the human-like parameter set, each original game character and the new game character are controlled to execute simulated game matches with 30 million samples to obtain the strength test results of the new game character.

[0209] In summary, in this embodiment, by training a target strategy model to control each game character and executing a large number of simulated game matches in a short period of time, a large amount of game battle data can be quickly obtained. This allows for the statistical analysis of the strength test results of the new game character, thereby improving the testing progress of the new game character. At the same time, by training strategy models for each original hero and setting corresponding anthropomorphic parameter sets for the target strategy model, it is ensured that when the target strategy model controls each game character, it more closely matches the operating tendencies and reaction speed of real players, thus ensuring the accuracy of the balance test for the new game character.

[0210] Furthermore, the game AI control method based on reinforcement learning training strategy model provided in this application is not limited to testing the balance of new game characters in MOBA games, but can also be used to test the balance of newly designed game equipment.

[0211] Based on the same technical concept, see [reference] Figure 15 As shown in the figure, this application embodiment also provides a game artificial intelligence control device 1500, which includes:

[0212] Grouping module 1501 is used to obtain multiple game character combinations based on selected original game characters and new game characters;

[0213] Processing module 1502 is used to execute game simulation matches based on multiple game character combinations to obtain corresponding game battle data. Specifically, when executing a game simulation match based on a single game character combination, the following operations are performed:

[0214] At each preset number of game frames, based on the game state information represented by the current game frame, corresponding initial action commands are obtained for each game character in at least one subsequent game frame.

[0215] Preset and adjust each initial action command to obtain the corresponding target action command, and use each target action command to control the corresponding game character to complete the combat action.

[0216] In one possible implementation, when the processing module 1502 obtains the corresponding initial action commands for each game character in at least one subsequent game frame based on the game state information represented by the current game frame, the processing module 1502 is used to:

[0217] Based on game state information, extract the background space features of the current game frame, the combat power features of each game character, and the game state features of the current game frame.

[0218] The background space features and game state features are respectively fused with the combat power features of each game character to obtain multiple target fusion features;

[0219] Based on the fusion features of multiple targets, corresponding initial action commands are obtained for each game character in at least one subsequent game frame.

[0220] In one possible implementation, when pre-adjusting each initial action command to obtain the corresponding target action command, the processing module 1502 is used to:

[0221] Anthropomorphic parameter sets are obtained based on the historical combat data of each original game character. These anthropomorphic parameter sets are used to control the delay state of combat actions.

[0222] Each initial action command is adjusted based on a human-like parameter set to obtain the corresponding target action command.

[0223] In one possible implementation, when obtaining the anthropomorphic parameter set based on the historical combat data of each original game character, the processing module 1502 is used for:

[0224] Based on the original game characters, multiple sample game character combinations were obtained, and sample game simulations were executed on each of the multiple sample game character combinations to obtain corresponding sample game battle data.

[0225] Based on the obtained game battle data from each sample, the strength test results for each original game character were obtained.

[0226] Based on the historical combat data of each original game character, the overall combat strength of each original game character is obtained;

[0227] Based on the strength test results of each original game character and their overall combat strength, a set of anthropomorphic parameters was obtained.

[0228] In one possible implementation, when executing simulated matches of sample games based on multiple combinations of sample game characters to obtain corresponding sample game battle data, the processing module 1502 is used for:

[0229] When performing a sample game simulation based on a sample game character combination, perform the following operations:

[0230] For each preset number of sample game frames, based on the game state information represented by the current sample game frame, for each game character in at least one subsequent sample game frame, multiple candidate sample commands are obtained, as well as the benefit evaluation values ​​corresponding to the multiple candidate sample commands. The benefit evaluation values ​​are used to measure the benefits brought to the sample game simulation game after the candidate sample commands are executed.

[0231] Based on the revenue evaluation values ​​corresponding to multiple candidate sample instructions, the weight matrix corresponding to the game state information represented by the current sample game frame is determined, and based on the weight matrix, the execution probability corresponding to each of the multiple candidate sample instructions is determined.

[0232] Based on the execution probabilities of multiple candidate sample instructions, target sample instructions that meet the preset probability requirements are selected for each game character, and each target sample instruction is used to control the corresponding game character to complete combat actions.

[0233] In one possible implementation, after obtaining the strength test results for each original game character based on the acquired sample game battle data, the processing module 1502 is further used for:

[0234] Based on the original and new game characters, multiple new sample game character combinations were obtained. Based on these new sample game character combinations, new sample game simulation matches were executed to obtain corresponding new sample game battle data.

[0235] Based on the newly obtained sample game battle data, the strength test change rate of the new game characters was obtained;

[0236] When the rate of change of the intensity test meets the preset convergence condition, it is determined that the control environment for artificial intelligence control of the new game character has reached a stable state.

[0237] In one possible implementation, the revenue assessment value includes at least one of the following:

[0238] The first sub-benefit evaluation value characterizing the experience and resource growth brought to the current game character by the candidate sample instruction;

[0239] The second sub-benefit evaluation value represents the kill and assist rewards that the candidate sample instruction brings to the current game character.

[0240] The third sub-profit evaluation value that characterizes the candidate sample instruction that results in the win or loss of the current sample simulation game.

[0241] In one possible implementation, before obtaining multiple combinations of game characters based on the pool of selected original and new game characters, the processing module 1502 is further used for:

[0242] Obtain the target version information of the game version to be launched, and update the execution environment of the game simulation based on the differences between the target version information and the historical version information corresponding to the running historical version.

[0243] In one possible implementation, after executing simulated game matches based on multiple game character combinations and obtaining the corresponding game battle data, the processing module 1502 is further used for:

[0244] Based on the obtained game battle data, the strength test results corresponding to the new game character are obtained. The strength test results corresponding to the new game character include the classification test results corresponding to multiple test indicators, and corresponding strength threshold ranges are preset for multiple test indicators.

[0245] In one possible implementation, after obtaining the strength test results for the new game character based on the acquired game battle data, the processing module 1502 is further used for:

[0246] The results of each category of the strength test are compared with their respective strength threshold ranges to obtain the comparison results.

[0247] If the comparison results indicate that the test results of each category do not exceed their respective intensity threshold range, then the new game character is determined to meet the preset balance requirements.

[0248] Having introduced the game artificial intelligence control method and apparatus according to exemplary embodiments of this application, we will now introduce a computer device according to another exemplary embodiment of this application.

[0249] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0250] Based on the same inventive concept as the above-described method embodiments, this application also provides a computer device. In one embodiment, the computer device may be a server, such as... Figure 1 The server 120 is shown. In this embodiment, the computer device 1600 is structured as follows: Figure 16 As shown, it may include at least a memory 1601, a communication module 1603, and at least one processor 1602.

[0251] The memory 1601 is used to store computer programs executed by the processor 1602. The memory 1601 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0252] Memory 1601 may be volatile memory, such as random-access memory (RAM); memory 1601 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1601 may be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1601 may be a combination of the above-described memories.

[0253] Processor 1602 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 1602 is used to implement the above-mentioned game artificial intelligence control method when calling computer programs stored in memory 1601.

[0254] The communication module 1603 is used to communicate with terminal devices and other servers.

[0255] This application embodiment does not limit the specific connection medium between the memory 1601, communication module 1603, and processor 1602. This application embodiment... Figure 16 The memory 1601 and the processor 1602 are connected via a bus 1604, and the bus 1604 is in Figure 16 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 1604 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 16 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0256] The memory 1601 stores a computer storage medium containing computer-executable instructions for implementing the game artificial intelligence control method of this application embodiment. The processor 1602 is used to execute the above-described game artificial intelligence control method.

[0257] In another embodiment, the computer device can also be other computer devices, such as... Figure 1 The terminal device 110 is shown. In this embodiment, the structure of the computer device can be as follows: Figure 17As shown, it includes components such as: communication component 1710, memory 1720, display unit 1730, camera 1740, sensor 1750, audio circuit 1760, Bluetooth module 1770, and processor 1780.

[0258] The communication component 1710 is used to communicate with the server. In some embodiments, it may include a Wireless Fidelity (WiFi) module, which is a short-range wireless transmission technology, and the electronic device can send and receive information through the WiFi module.

[0259] The memory 1720 can be used to store software programs and data. The processor 1780 executes various functions of the terminal device 110 and performs data processing by running the software programs or data stored in the memory 1720. The memory 1720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 1720 stores an operating system that enables the terminal device 110 to run. In this application, the memory 1720 can store the operating system and various applications, and may also store a computer program that executes the game artificial intelligence control method of the embodiments of this application.

[0260] The display unit 1730 can also be used to display information input by the object or information provided to the object, as well as a graphical user interface (GUI) of various menus of the terminal device 110. Specifically, the display unit 1730 may include a display screen 1732 disposed on the front of the terminal device 110. The display screen 1732 may be configured as a liquid crystal display, a light-emitting diode, or the like. The display unit 1730 can be used to display game simulation interfaces, etc., as described in the embodiments of this application.

[0261] The display unit 1730 can also be used to receive input digital or character information and generate signal inputs related to object settings and function control of the terminal device 110. Specifically, the display unit 1730 may include a touch screen 1731 disposed on the front of the terminal device 110, which can collect touch operations on or near the object, such as clicking a button, dragging a scroll bar, etc.

[0262] The touchscreen 1731 can be placed on top of the display screen 1732, or the touchscreen 1731 and the display screen 1732 can be integrated to realize the input and output functions of the terminal device 110. After integration, it can be referred to as a touch display screen. In this application, the display unit 1730 can display the application and the corresponding operation steps.

[0263] Camera 1740 can be used to capture still images, and objects can publish images captured by camera 1740 through an application. There can be one or multiple cameras 1740. An optical image of an object is generated through a lens and projected onto a photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to processor 1780 to be converted into a digital image signal.

[0264] The terminal device may also include at least one sensor 1750, such as an accelerometer 1751, a proximity sensor 1752, a fingerprint sensor 1753, and a temperature sensor 1754. The terminal device may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, and motion sensor.

[0265] Audio circuitry 1760, speaker 1761, and microphone 1762 provide an audio interface between the device and terminal device 110. Audio circuitry 1760 converts received audio data into electrical signals, which are then transmitted to speaker 1761, where they are converted into sound signals for output. Terminal device 110 may also be equipped with volume buttons for adjusting the volume of the sound signal. On the other hand, microphone 1762 converts collected sound signals into electrical signals, which are received by audio circuitry 1760, converted into audio data, and then output to communication component 1710 for transmission to, for example, another terminal device, or to memory 1720 for further processing.

[0266] The Bluetooth module 1770 is used to interact with other Bluetooth devices that also have a Bluetooth module via the Bluetooth protocol. For example, a terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smartwatch) that also has a Bluetooth module through the Bluetooth module 1770, thereby exchanging data.

[0267] The processor 1780 is the control center of the terminal device, connecting various parts of the terminal through various interfaces and lines. It executes various functions and processes data by running or executing software programs stored in the memory 1720 and calling data stored in the memory 1720. In some embodiments, the processor 1780 may include one or more processing units; the processor 1780 may also integrate an application processor and a baseband processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the baseband processor mainly handles wireless communication. It is understood that the baseband processor may not be integrated into the processor 1780. In this application, the processor 1780 can run the operating system, applications, user interface display and touch response, as well as the game artificial intelligence control method of this embodiment. Furthermore, the processor 1780 is coupled to the display unit 1730.

[0268] Furthermore, it should be noted that in the specific embodiments of this application, object data related to game artificial intelligence control is involved. When the above embodiments of this application are applied to specific products or technologies, permission or consent from the object is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0269] In some possible implementations, various aspects of the game artificial intelligence control method provided in this application can also be implemented in the form of a program product, which includes a computer program that, when run on a computer device, causes the computer device to perform the steps in the game artificial intelligence control method according to the various exemplary embodiments of this application described above.

[0270] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0271] The program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include a computer program, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0272] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0273] Computer programs contained on readable media may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0274] Computer programs for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The computer program can execute entirely on the user's computer device, partially on the user's computer device, as a standalone software package, partially on the user's computer device and partially on a remote computer device, or entirely on a remote computer device. In cases involving remote computer devices, the remote computer device can be connected to the user's computer device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer device (e.g., via the Internet using an Internet service provider).

[0275] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0276] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0277] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0278] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0279] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0280] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0281] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for controlling artificial intelligence in games, characterized in that, include: Based on the selected original game characters and new game characters, multiple combinations of game characters can be obtained; Based on the aforementioned combinations of game characters, simulated game matches are executed to obtain corresponding game battle data. Specifically, when executing a simulated game match based on a single combination of game characters, the following operations are performed: At each preset number of game frames, based on the game state information represented by the current game frame, corresponding initial action commands are obtained for each game character in at least one subsequent game frame. Based on the original game characters, multiple sample game character combinations are obtained. Simulated matches are then executed on each of these combinations to obtain corresponding sample game battle data. Based on this battle data, strength test results for each original game character are obtained. According to the historical combat data of each original game character, their overall combat strength is determined. Based on the strength test results and overall combat strength of each original game character, a set of anthropomorphic parameters is obtained. This set controls the delay state of combat actions. Initial action commands are adjusted based on the anthropomorphic parameter set to obtain corresponding target action commands. These target action commands are then used to control the corresponding game characters to complete combat actions.

2. The method as described in claim 1, characterized in that, The game state information represented by the current game frame is used to obtain corresponding initial action commands for each game character in at least one subsequent game frame, including: Based on the game state information, the background space features of the current game frame, the combat power features of each game character, and the game state features of the current game frame are extracted. The background space features and the game state features are respectively fused with the combat power features of each game character to obtain multiple target fusion features; Based on the multiple target fusion features, corresponding initial action commands are obtained for each game character in at least one subsequent game frame.

3. The method as described in claim 1, characterized in that, The process involves executing simulated matches of the sample games based on the multiple sample game character combinations to obtain corresponding sample game battle data, including: When performing a sample game simulation based on a sample game character combination, perform the following operations: At each interval of the preset number of sample game frames, based on the game state information represented by the current sample game frame, for each game character in at least one subsequent sample game frame, a number of corresponding candidate sample instructions are obtained, as well as the benefit evaluation value corresponding to the number of candidate sample instructions. The benefit evaluation value is used to measure the benefit brought to the sample game simulation game after the candidate sample instructions are executed. Based on the revenue evaluation values ​​corresponding to the multiple candidate sample instructions, a weight matrix corresponding to the game state information represented by the current sample game frame is determined, and based on the weight matrix, the execution probability corresponding to each of the multiple candidate sample instructions is determined. Based on the execution probabilities of the multiple candidate sample instructions, target sample instructions that meet the preset probability requirements are selected for each game character, and each target sample instruction is used to control the corresponding game character to complete the combat action.

4. The method as described in claim 1, characterized in that, After obtaining the strength test results for each original game character based on the acquired sample game battle data, the process further includes: Based on the original game characters and the new game characters, multiple new sample game character combinations are obtained, and based on the multiple new sample game character combinations, new sample game simulation matches are executed to obtain corresponding new sample game battle data. Based on the newly obtained sample game battle data, the strength test change rate of the new game character was obtained; When the rate of change of the intensity test meets the preset convergence condition, it is determined that the control environment for artificial intelligence control of the new game character has reached a stable state.

5. The method as described in claim 3, characterized in that, The revenue assessment value includes at least one of the following: The first sub-benefit evaluation value characterizing the experience and resource growth brought to the current game character by the candidate sample instruction; The second sub-benefit evaluation value represents the kill and assist rewards that the candidate sample instruction brings to the current game character. The third sub-profit evaluation value characterizes the outcome of the current sample game simulation match caused by the candidate sample instruction.

6. The method as described in claim 1 or 2, characterized in that, Before obtaining multiple game character combinations based on the selected original game characters and new game characters, the process also includes: Obtain the target version information of the game version to be launched, and update the execution environment of the game simulation game based on the difference between the target version information and the historical version information corresponding to the historical version in operation.

7. The method as described in claim 1 or 2, characterized in that, After obtaining the corresponding game battle data by executing game simulation matches based on the multiple game character combinations, the process further includes: Based on the obtained game battle data, the strength test results corresponding to the new game character are obtained. The strength test results corresponding to the new game character include the classification test results corresponding to multiple test indicators, and corresponding strength threshold ranges are preset for the multiple test indicators. After obtaining the strength test results corresponding to the new game character based on the acquired game battle data, the process further includes: The strength test results are compared with their respective strength threshold ranges to obtain comparison results. If the comparison results indicate that the test results of each category do not exceed their respective intensity threshold range, then the new game character is determined to meet the preset balance requirements.

8. A game artificial intelligence control device, characterized in that, include: The grouping module is used to obtain multiple combinations of game characters based on selected original and new game characters; The processing module is used to execute simulated game matches based on the multiple game character combinations to obtain corresponding game battle data. Specifically, when executing a simulated game match based on a single game character combination, the following operations are performed: At each preset number of game frames, based on the game state information represented by the current game frame, corresponding initial action commands are obtained for each game character in at least one subsequent game frame. Based on the original game characters, multiple sample game character combinations are obtained. Simulated matches are then executed on each of these combinations to obtain corresponding sample game battle data. Based on this battle data, strength test results for each original game character are obtained. According to the historical combat data of each original game character, their overall combat strength is determined. Based on the strength test results and overall combat strength of each original game character, a set of anthropomorphic parameters is obtained. This set controls the delay state of combat actions. Initial action commands are adjusted based on the anthropomorphic parameter set to obtain corresponding target action commands. These target action commands are then used to control the corresponding game characters to complete combat actions.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.

11. A computer program product, characterized in that, When the computer program product is invoked by a computer, it causes the computer to perform the method as described in any one of claims 1-7.

Citation Information

Patent Citations

  • Game character creating method, game character creating device, electronic equipment and storage medium

    CN109794065A

  • Control method and device for virtual character in game and computer equipment

    CN113996063A

  • Decision model training method and device, computer equipment and storage medium

    CN114404976A