Intelligent agent using method and device, computer equipment and storage medium
By training operation prediction models and dynamically adjusting strategies, the problem of fixed agent operations is solved, improving its adaptability and decision-making quality in the game, and improving the game experience.
Patent Information
- Application Number
- CN202411418105.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2025-02-21
AI Technical Summary
The existing agents have fixed equipment purchases, skill upgrades and other operations in the game game, and cannot be adjusted according to changes in the battlefield within the game, resulting in human players being able to understand their behavior patterns and affect the player's experience.
By obtaining multiple historical operation data and historical gold coin information, training operation prediction models, predicting and performing equipment purchases, skill upgrades and other operations of the agent, dynamically adjusting strategies to adapt to the current situation.
It improves the dynamic adaptability and decision-making quality of the agent, makes its behavior more difficult to predict, improves the diversity and entertainment of the game, and improves the player's experience.
Smart Images
Figure CN119488719A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for using an intelligent entity. Background Art
[0002] In recent years, with the continuous development of the game industry, the design of game agents based on artificial intelligence (AI) has become an indispensable part of game development. The performance of AI-trained agents in competitive battles is quite impressive, but at the same time, these agents still have some minor flaws. For example, the purchase of equipment and skill upgrades in the game are still implemented in a fixed way and cannot be adjusted according to the changes in the battlefield. In this way, some experienced human players can see the fixed behavior patterns of AI through some clues, which affects the player experience.
[0003] Therefore, there is an urgent need for a method of using intelligent agents to realize the intelligence of the skills and equipment of intelligent agents, so as to enhance the strength of the intelligent agents and their anthropomorphic capabilities, thereby enhancing the use value of the intelligent agents in the game. Summary of the invention
[0004] This application provides a method, device, equipment and storage medium for using an intelligent agent, aiming to solve the problem that the current intelligent agent's equipment purchase and skill upgrade in the game are still implemented in a fixed way and cannot be adjusted according to the changes in the battlefield in the game. In this way, some experienced human players can gain insight into the fixed behavior pattern of AI through some clues, thus affecting the player's experience.
[0005] In a first aspect, the present application provides a method for using an intelligent agent, comprising:
[0006] Obtain multiple historical operation data corresponding to a preset game environment and historical gold coin information corresponding to each historical operation data;
[0007] Obtaining the historical operations corresponding to each game data; wherein the historical operations include equipment purchase operations, equipment sale operations, skill upgrade operations, and any one of other operations;
[0008] Inputting a plurality of historical operation data and historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained, and the operation prediction model outputs a predicted operation;
[0009] The operation prediction model is trained according to the predicted operation and the corresponding historical operation, and the trained operation prediction model is obtained;
[0010] In the game environment, obtain the current equipment information, current skill information and current gold coin information corresponding to the intelligent agent to be used;
[0011] Input the current equipment information, current skill information and current gold coin information into the prediction operation model, and the prediction operation model outputs the usage information corresponding to the agent; wherein the usage information includes one or more of equipment purchase operation, equipment sale operation, skill upgrade operation and other operations;
[0012] Complete the use of the agent in the game environment based on the usage information.
[0013] In some embodiments, the historical gold coin information includes the first historical gold coin of the game character corresponding to the historical operation data; before obtaining the historical operation corresponding to each game data, it also includes: obtaining equipment information corresponding to the game environment; the equipment information includes multiple equipment and the equipment price corresponding to each equipment; obtaining the minimum equipment price among multiple equipment prices; if the first historical gold coin corresponding to the historical operation data is less than the minimum equipment price, determining that the historical operation data is redundant operation data; removing redundant operation data from multiple historical operation data.
[0014] In some embodiments, before inputting multiple historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained, it also includes: dividing each historical operation data into positive sample data and negative sample data according to the historical operation, wherein the historical operation corresponding to the positive sample data includes any one of equipment purchase operation, equipment sale operation and skill upgrade operation, and the historical operation corresponding to the negative sample data includes other operations; respectively inputting the positive sample data, the historical gold coin information corresponding to the positive sample data and the negative sample data, the historical gold coin information corresponding to the negative sample data into the operation prediction model.
[0015] Exemplarily, before respectively inputting the positive sample data, the historical gold coin information corresponding to the positive sample data, and the negative sample data, the historical gold coin information corresponding to the negative sample data into the operation prediction model, it also includes: adjusting the number of negative sample data so that the ratio of the number of negative sample data to the number of positive sample data is a preset ratio.
[0016] In some embodiments, before inputting multiple historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained, it also includes: determining the historical operation data of the equipment selling operation as the target data; copying the target data according to a preset number of times to generate multiple backup data; wherein the historical gold coin information corresponding to the backup data is the same as the target data, so as to input multiple backup data, the historical gold coin information corresponding to each backup data, multiple historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained.
[0017] In some embodiments, the historical gold coin information includes the first historical gold coins of the game character corresponding to the historical operation data and the second historical gold coins of the game character and the opposing character in the game environment; multiple historical operation data and the historical gold coin information corresponding to each historical operation data are input into the operation prediction model to be trained, including: obtaining the first combat power information of the game character and the second combat power information of the opposing character corresponding to each historical operation data; inputting the historical operation data and the first historical gold coins, the second historical gold coins, the first combat power information and the second combat power information corresponding to each historical operation data into the operation prediction model to be trained.
[0018] In some embodiments, after completing the use of the intelligent agent in the game environment according to the usage information, it also includes: obtaining state information of the intelligent agent and the opposing character corresponding to the intelligent agent in the game environment; obtaining difficulty information corresponding to the game environment; and completing the optimization of the operation prediction model according to the state information and difficulty information.
[0019] In a second aspect, the present application further provides an intelligent agent using device, comprising:
[0020] An information acquisition unit, used to acquire a plurality of historical operation data corresponding to a preset game environment and historical gold coin information corresponding to each historical operation data;
[0021] An operation acquisition unit, used to acquire the historical operation corresponding to each game data; wherein the historical operation includes any one of an equipment purchase operation, an equipment sale operation, a skill upgrade operation and other operations;
[0022] A prediction output unit, used to input a plurality of historical operation data and historical gold coin information corresponding to each historical operation data into an operation prediction model to be trained, and the operation prediction model outputs a predicted operation;
[0023] A training completion unit, used to complete the training of the operation prediction model according to the predicted operation and the corresponding historical operation, and obtain the trained operation prediction model;
[0024] A current acquisition unit, used to acquire current equipment information, current skill information and current gold coin information corresponding to the intelligent agent to be used in the game environment;
[0025] The usage output unit is used to input the current equipment information, current skill information and current gold coin information into the prediction operation model, and the prediction operation model outputs the usage information corresponding to the intelligent agent; wherein the usage information includes one or more of equipment purchase operation, equipment sale operation, skill upgrade operation and other operations;
[0026] The usage completion unit is used to complete the use of the intelligent agent in the game environment according to the usage information.
[0027] In a third aspect, the present application further provides a computer device, including:
[0028] Memory and processor;
[0029] The memory is used to store computer programs;
[0030] The processor is used to execute the computer program and implement the steps of the intelligent agent use method as described in the first aspect above when executing the computer program.
[0031] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the method for using the intelligent agent as described in the first aspect above.
[0032] The provided method first collects multiple historical operation data and the corresponding historical gold information. These data come from previous games in similar game environments. The data recorded for each historical operation includes multiple operation types such as equipment purchase, equipment sale, skill upgrade, etc. The collected historical operation data and gold information are input into an operation prediction model to be trained, which is designed to output the prediction results of future operations. By constantly comparing the predicted operations with the actual historical operations, the model gradually adjusts its own parameters so that it can learn effective strategies and decisions. In the actual game, the system obtains the current equipment, skills and gold information of the agent in real time. This real-time information is input into the trained operation prediction model to predict the next appropriate operation (such as equipment purchase, skill upgrade, etc.). According to the suggestions output by the model, the agent can dynamically adjust its behavior and strategy to complete the corresponding game operation.
[0033] The provided method has at least the following beneficial effects:
[0034] 1. Enhanced dynamic adaptability: Through the learning ability of the model, the agent can adjust its behavior according to the changes in the current game situation, thereby avoiding the defects of fixed strategies and improving its adaptability to complex environments.
[0035] 2. Difficult to predict and counter: It is difficult for human players to predict the behavior patterns of AI through historical games, because the actions of the intelligent agent change according to the real-time environment and dynamic decision-making, and are no longer simple presets or rules.
[0036] 3. Improve user experience: The dynamic strategy of the intelligent agent makes the game more diverse and challenging, improves the overall entertainment and competitiveness of the game, and improves the player's gaming experience.
[0037] 4. Better decision quality: Through deep learning of historical data and strategy optimization, the agent can make better decisions and achieve better performance in the game.
[0038] In general, the method provided can allow the agent to use different outfit styles in different games through the training and use of the prediction model, making the agent more human-like, and allowing the agent to decide the order of upgrading skills and equipment in the game. At the same time, the equipment replacement scheme proposed in this patent allows the agent to have higher strength in the later stages of the game. This enables the game agent to better adapt to the real-time environment, form flexible and changeable strategies, and bring a more challenging and interesting game experience.
[0039] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0041] Figure 1 is a schematic flow chart of the steps of a method for using an intelligent agent provided in one embodiment of the present application;
[0042] Figure 2 It is a schematic diagram of the principle of equipment synthesis provided by an embodiment of the present application;
[0043] Figure 3 It is a schematic diagram of a characteristic structure provided by an embodiment of the present application;
[0044] Figure 4 is a schematic diagram of the network structure of the operation prediction model provided in the embodiment of the present application;
[0045] Figure 5 is a schematic diagram of the structure of an intelligent agent using device provided in an embodiment of the present application;
[0046] Figure 6 It is a schematic block diagram of the structure of the computer device provided in the embodiment of the present application.
[0047] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0049] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0050] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0051] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0052] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0053] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0054] In recent years, with the continuous development of the game industry, the design of game agents based on artificial intelligence (AI) has become an indispensable part of game development. The performance of AI-trained agents in competitive battles is quite impressive, but at the same time, these agents still have some minor flaws. For example, the purchase of equipment and skill upgrades in the game are still implemented in a fixed way and cannot be adjusted according to the changes in the battlefield. In this way, some experienced human players can see the fixed behavior patterns of AI through some clues, which affects the player experience.
[0055] Therefore, there is an urgent need for a method of using intelligent agents to realize the intelligence of the skills and equipment of intelligent agents, so as to enhance the strength of the intelligent agents and their anthropomorphic capabilities, thereby enhancing the use value of the intelligent agents in the game.
[0056] To solve the above problems, please refer to Figure 1 , Figure 1 1 is a schematic flow chart of an agent use method provided by an embodiment of the present application. The agent use method can be implemented by a computer device, and the computer device can be deployed on a single server or a server cluster. It can also be deployed on a handheld terminal, a laptop computer, a wearable device or a robot, etc.
[0057] At the same time, if Figure 2 As shown, the existing equipment purchase method is mainly to pre-configure the purchased large equipment through scripts. Each large equipment is synthesized from many small pieces, so a synthesis tree will be generated for each large equipment, such as Figure 2 Equipment A is composed of B and C, and B is composed of D and E. When purchasing, start from the top of the synthesis tree and traverse the layers downwards. If there are enough gold coins to buy a piece of equipment on the leaf node, then buy this equipment, and do not buy the equipment that has already been purchased. Skill upgrades are also upgraded in a fixed upgrade order.
[0058] Furthermore, the main disadvantage of the existing method is that the equipment and skill upgrades are fixed. For example, in many competitive games, the damage is divided into magic damage and physical damage, so when laning, the protective equipment will target the laning hero. At the same time, some heroes have the attribute of swing positions. In some lineups with sufficient damage, they will be equipped with meat to act as the front row, and in some lineups lacking damage, they need to be equipped with output equipment to output. When the game reaches the later stage, it may be necessary to sell some equipment and replace it with equipment with higher returns. The order of skill upgrades also has the same problem. A hero will inevitably have different main skills in the face of different situations. Therefore, the existing method is difficult to meet the diversity and flexibility of players' equipment and skill upgrades.
[0059] like Figure 1 As shown, the agent use method provided in this embodiment includes steps S101 to S107. The details are as follows:
[0060] Step S101. Obtain multiple historical operation data corresponding to a preset game environment and historical gold coin information corresponding to each historical operation data.
[0061] Specifically, by collecting data from games that have been played in a pre-set game environment. Each record includes player actions at different time points, as well as corresponding game environment conditions (such as the number of gold coins). For example, data is extracted from the historical logs of the game server, and player operations from multiple different games and the gold coin status at the time are collected. These data are stored in a database or data warehouse for subsequent processing. Then, by collecting rich and diverse game behavior data, the basis for training the model is formed to ensure that the model has a comprehensive cognitive and decision-making basis.
[0062] Step S102. Obtain the historical operations corresponding to each game data; wherein the historical operations include equipment purchase operations, equipment sale operations, skill upgrade operations and any one of other operations.
[0063] Specifically, by extracting the specific operations performed by the player in the game from the data obtained in step S101, such as equipment purchase, equipment sale, skill upgrade, etc., a script or tool is used to analyze the history record file, parse and structure different types of operation behaviors. Through fine-grained operation records, the model can identify and learn the best strategies in different situations.
[0064] In some embodiments, the historical gold coin information includes the first historical gold coin of the game character corresponding to the historical operation data; before obtaining the historical operation corresponding to each game data, it also includes: obtaining equipment information corresponding to the game environment; the equipment information includes multiple equipment and the equipment price corresponding to each equipment; obtaining the minimum equipment price among multiple equipment prices; if the first historical gold coin corresponding to the historical operation data is less than the minimum equipment price, determining that the historical operation data is redundant operation data; removing redundant operation data from multiple historical operation data.
[0065] By obtaining the first historical gold coin amount of the character corresponding to each historical operation data. This clarifies the initial gold coin state of the character before performing a series of operations. Before processing the historical operation data, the method also obtains detailed information on all possible equipment, including the price of each equipment. This information can usually be extracted from the game database or configuration information. At the same time, by traversing the prices of all equipment, find the minimum value. This serves as a benchmark to determine whether the player can purchase equipment at a certain moment. The first historical gold coin before detecting each historical operation data. If this number is less than the minimum equipment price, the operation data is considered redundant (because no equipment purchase operation can be performed in this state). Remove these redundant data from multiple historical operation data to ensure that subsequent analysis and model training are more focused on actual and effective player behavior.
[0066] After removing redundant operation data, the retained data is more representative and practical, which helps the model learn actual player behavior and decision-making logic more accurately. At the same time, it reduces the processing of invalid data, reduces the complexity of the data set, improves the efficiency of model training, and may increase the convergence speed of the model. Because the data used in the training process is more relevant and accurate, the model prediction performance will be enhanced due to better data quality, improving the actual performance in the game environment. And by avoiding the processing and storage of useless data, the system can use computing and storage resources more efficiently.
[0067] In summary, the above embodiments clean up the data set to ensure that only valid data is used during model training, thereby improving the accuracy and efficiency of model prediction. This process is crucial in the data preprocessing stage of machine learning, especially in dynamic and complex game environments.
[0068] Step S103: Input multiple historical operation data and historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained, and the operation prediction model outputs the predicted operation.
[0069] Specifically, by using historical operations and gold coin data as input, an operation prediction model (such as a neural network or decision tree) is trained in order to predict future operations. For example, a machine learning framework (such as TensorFlow or PyTorch) is used to process data input, build and train the model, and output possible predicted actions. This enables the model to predict future operations based on historical behaviors and real-time conditions.
[0070] Step S104: Complete the training of the operation prediction model according to the predicted operation and the corresponding historical operation, and obtain the trained operation prediction model.
[0071] Specifically, by comparing the predicted operations output by the model with the actual historical operations, the model parameters are optimized and adjusted until the model's prediction accuracy for historical data reaches an acceptable level. Back propagation and gradient descent methods are used to adjust the model weights and perform multiple rounds of iterative training to improve the model accuracy. This ensures that the model can efficiently predict and select the optimal operations in future games, thereby improving the performance of the agent.
[0072] In some embodiments, before inputting multiple historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained, it also includes: dividing each historical operation data into positive sample data and negative sample data according to the historical operation, wherein the historical operation corresponding to the positive sample data includes any one of equipment purchase operation, equipment sale operation and skill upgrade operation, and the historical operation corresponding to the negative sample data includes other operations; respectively inputting the positive sample data, the historical gold coin information corresponding to the positive sample data and the negative sample data, the historical gold coin information corresponding to the negative sample data into the operation prediction model.
[0073] By dividing each historical operation data into positive sample data and negative sample data according to the historical operation type. Positive sample data includes strategically important operations, such as equipment purchase, equipment sale, skill upgrade, etc. Negative sample data includes other less important or frequent operations. By inputting positive sample data, its corresponding historical gold information, and negative sample data and its corresponding historical gold information into the operation prediction model respectively. This classification input helps the model better identify and weigh the importance of different types of operations.
[0074] By focusing on more strategic operations, the model can more clearly learn the characteristics and impact of these key operations, thereby improving prediction accuracy and effectiveness. The model can optimize operations with higher strategic value in the game (such as equipment purchases or skill upgrades) to improve the decision-making performance of the agent. At the same time, by classifying and organizing training data, the interference of noise data can be reduced and the speed at which the model learns effective patterns can be accelerated.
[0075] Exemplarily, before respectively inputting the positive sample data, the historical gold coin information corresponding to the positive sample data, and the negative sample data, the historical gold coin information corresponding to the negative sample data into the operation prediction model, it also includes: adjusting the number of negative sample data so that the ratio of the number of negative sample data to the number of positive sample data is a preset ratio.
[0076] By adjusting the number of negative sample data before inputting data into the model, the ratio of the number of negative samples to positive samples reaches a preset ratio to ensure the rationality and balance of sample distribution. By adjusting the number of samples, the influence of negative samples is limited so that positive samples are not too sparse, thereby promoting balanced learning of the model and avoiding bias towards frequent but unimportant operations. By setting a reasonable positive-negative sample ratio (such as the number of positive samples: the number of negative samples = 1 / 2 to 1 / 3), the model bias caused by sample imbalance is reduced, and the sensitivity of the model to important operations is improved. Since the preset ratio can be set according to the specific game environment and requirements, it can ensure that there is enough information during model training, and at the same time, it does not rely too much on a certain type of sample, thereby improving the generalization ability of the model. Furthermore, the above-mentioned embodiments and examples thereof enhance the training efficiency and decision-making quality of the model through the classification and quantity adjustment of samples, making the intelligent agent more competitive and adaptable.
[0077] In some embodiments, before inputting multiple historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained, it also includes: determining the historical operation data of the equipment selling operation as the target data; copying the target data according to a preset number of times to generate multiple backup data; wherein the historical gold coin information corresponding to the backup data is the same as the target data, so as to input multiple backup data, the historical gold coin information corresponding to each backup data, multiple historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained.
[0078] In the historical operation data set, there are too few samples of equipment selling operations, and the traditional intelligent agents in the later stage may not be able to sell and replace equipment according to the situation after the equipment is fully equipped. For this reason, this application identifies the records containing equipment selling operations and defines these records as target data. This choice may be because the equipment selling operations are relatively small and have a significant impact on decision-making, requiring higher model analysis accuracy. For each target data, multiple backups (such as 2 to 3 times) are generated according to the preset replication multiples. Each backup retains the same historical gold coin information and other related data as the original target data. This process is equivalent to data enhancement, especially for equipment selling operations.
[0079] By inputting the generated backup data together with their historical gold information, as well as the original operation data and its historical gold information, into the operation prediction model to be trained. This integrated training set helps the model to more comprehensively understand and learn the importance of different operations (especially equipment sales) and the corresponding environmental conditions. At the same time, by replicating the equipment sales operation data, the proportion of such operations in the training set is increased, which effectively solves the common category imbalance problem and prevents the model from being biased towards frequent categories. Better representation of equipment sales operations in the model enables the model to better handle equipment sales scenarios in real game operations and improve decision-making capabilities. At the same time, expanding data samples of specific categories can help the model capture the characteristics of these decisions more accurately, improve training accuracy and shorten convergence time. Strengthening the representation of equipment sales operations in the model helps to more deeply analyze the impact of equipment sales on overall tactics and strategies, and enhance the global decision-making ability of the intelligent agent.
[0080] In summary, the above embodiments improve the understanding and response capabilities of the operation prediction model to equipment selling operations through data supplementation and enhancement, so that the intelligent agent can adapt to various game scenarios more efficiently.
[0081] In some embodiments, the historical gold coin information includes the first historical gold coins of the game character corresponding to the historical operation data and the second historical gold coins of the game character and the opposing character in the game environment; multiple historical operation data and the historical gold coin information corresponding to each historical operation data are input into the operation prediction model to be trained, including: obtaining the first combat power information of the game character and the second combat power information of the opposing character corresponding to each historical operation data; inputting the historical operation data and the first historical gold coins, the second historical gold coins, the first combat power information and the second combat power information corresponding to each historical operation data into the operation prediction model to be trained.
[0082] In addition to recording the first historical gold coin information of the game character (player), the method also collects the second historical gold coin information of the opposing character. This provides a more comprehensive perspective on the economic status, including the economic strength of the player and his direct opponent. By obtaining the first combat power information of the game character related to each historical operation data and the second combat power information of the opposing character. Combat power information may include on-site capability indicators such as the character's current level, equipment bonus, skill upgrade status, etc. The expanded information (i.e., historical operation data, first and second historical gold coins, first and second combat power information) is input into the operation prediction model to be trained. This helps the model consider more complex and strategic decision factors when predicting.
[0083] Then, by combining the economic and combat power information of the opposing characters in model training, the model can make more targeted and strategic decisions, reflecting a more realistic game confrontation scenario. With richer information input, the model can better reflect the impact of multiple factors changing in the game on operational decisions, improving the model's ability to adapt to complex and dynamic environments. By utilizing real-time confrontation information, the intelligent agent can more efficiently predict and respond to the opponent's possible strategies and enhance its competitive advantage. In addition, the model takes into account the impact of both gold coins and combat power in the decision-making process, which helps to balance the relationship between short-term economic behavior and long-term combat capability development, making decisions more mature.
[0084] In summary, the above embodiments construct a more three-dimensional and accurate operation prediction framework by integrating history, information about opposing characters, and combat power factors, thereby improving the response strategy and expressiveness of the intelligent agent in an unpredictable game environment.
[0085] For example, in a common MOBA game, hundreds of equipment and 3 to 4 skills are designed. The provided method uses two features to represent all equipment, namely, how many pieces of equipment have been purchased (such as 0 to 5) and whether the current gold coins can be used to purchase the equipment. Two features are also used to represent each skill, namely, the current level of the skill and whether the skill can be upgraded. Together with the hero ID, faction ID, the remaining gold coins and total gold coins of the agent at the time node corresponding to the data constitute the first historical gold coins, and the remaining gold coins and total gold coins of the opposing character at the same time node constitute the second historical gold coins. The final feature representation is as follows: Figure 3 The comparison of total gold, remaining gold and combat power of both sides (such as attribute values and levels, which constitute different combat power situations according to different games) is added mainly to allow the model to perceive the advantages and disadvantages of both sides in the game, so as to make more targeted equipment.
[0086] In some embodiments, the structure diagram of the prediction operation model is as follows Figure 4 As shown in the figure, the model includes three fully connected layers (fc layers) that can combine the input features to correspond to the action, that is, 0, 1, 2 and 3 are used to represent not performing the action, buying equipment, selling equipment and upgrading skills respectively. At the same time, the specific upgraded skills or purchased / sold equipment can be output as equipment sublable and skill sublable. This greatly improves the accuracy of the model's prediction of the agent's operation.
[0087] Step S105. In the game environment, obtain the current equipment information, current skill information and current gold coin information corresponding to the intelligent agent to be used.
[0088] Specifically, by extracting the current agent information from the real-time state of the game, including equipment, skills, and available gold coins. For example, when the game is running, use the interface or API to obtain the current state of the agent and organize it into an input format. This provides sufficient real-time environmental information to support the next prediction operation and ensure that the agent operation meets the current game requirements.
[0089] Step S106. Input the current equipment information, current skill information and current gold coin information into the predictive operation model, and the predictive operation model outputs the usage information corresponding to the intelligent agent; wherein the usage information includes one or more of equipment purchase operation, equipment sale operation, skill upgrade operation and other operations.
[0090] Specifically, the model is used to process real-time input and output recommended actions, such as which equipment to buy or sell, or which skills should be upgraded. For example, the trained model is called in real time, and after the current state is input, the output suggestions are obtained, and the output operations are fine-tuned according to the strategy. This ensures that the intelligent agent can make flexible and efficient decisions in complex and changing game environments, and enhances adaptability and unpredictability.
[0091] Step S107. Complete the use of the intelligent agent in the game environment according to the use information.
[0092] Specifically, through the operation suggestions based on the model output, the agent performs specific operations in the game, such as purchasing equipment or upgrading skills. For example, the output of step S106 is applied to the game control logic, and the actual game operation is completed through instructions. The performance of the agent in the game can be improved. Through continuous learning and adaptation, it can show more intelligent and diverse game behaviors and improve the game experience. Through this set of strategies, the agent can adapt to the changing game environment in real time, optimize its decision-making process, make it difficult for human players to predict, and thus enhance the challenge and fairness of the game.
[0093] In some embodiments, after completing the use of the intelligent agent in the game environment according to the usage information, it also includes: obtaining state information of the intelligent agent and the opposing character corresponding to the intelligent agent in the game environment; obtaining difficulty information corresponding to the game environment; and completing the optimization of the operation prediction model according to the state information and difficulty information.
[0094] After the interaction between the agent and the opposing character is completed, its status information is obtained. The status information may include current health, magic value, equipment status, position, level, etc., reflecting the current game status of the agent and the opposing character. Extract the difficulty information of the current game environment. This may involve the overall difficulty level of the game, such as the complexity of the enemy AI, level settings, difficulty in obtaining resources, etc., which can affect the decision-making of players or agents. According to the obtained status information and difficulty information, adjust the parameters or strategies of the operation prediction model so that it can make more appropriate decisions in different environments and difficulties. This optimization process can include adjusting model weights, updating strategy functions, etc.
[0095] Then, by dynamically adjusting the model, the agent can respond more intelligently based on the specific conditions of the current environment (such as difficulty level and real-time status), improving the flexibility and adaptability of the agent. By optimizing the model with more detailed status and environmental information, the agent can more accurately predict the best operation and improve the accuracy and efficiency of its actual performance in the game. Collecting feedback and optimizing during the actual game process is equivalent to an online learning or reinforcement learning process, which gradually improves the decision-making ability of the agent. The optimized model can provide smoother and more intelligent game interactions, improve the user experience and game challenges, and make the game environment more attractive.
[0096] In summary, the above embodiments enable the intelligent agent to better adapt to the changing game environment by performing real-time status information collection and difficulty-based model optimization after game operations, thereby improving its decision-making ability and overall game performance.
[0097] The provided method first collects multiple historical operation data and the corresponding historical gold information. These data come from previous games in similar game environments. The data recorded for each historical operation includes multiple operation types such as equipment purchase, equipment sale, skill upgrade, etc. The collected historical operation data and gold information are input into an operation prediction model to be trained, which is designed to output the prediction results of future operations. By constantly comparing the predicted operations with the actual historical operations, the model gradually adjusts its own parameters so that it can learn effective strategies and decisions. In the actual game, the system obtains the current equipment, skills and gold information of the agent in real time. This real-time information is input into the trained operation prediction model to predict the next appropriate operation (such as equipment purchase, skill upgrade, etc.). According to the suggestions output by the model, the agent can dynamically adjust its behavior and strategy to complete the corresponding game operation.
[0098] The provided method has at least the following beneficial effects:
[0099] 1. Enhanced dynamic adaptability: Through the learning ability of the model, the agent can adjust its behavior according to the changes in the current game situation, thereby avoiding the defects of fixed strategies and improving its adaptability to complex environments.
[0100] 2. Difficult to predict and counter: It is difficult for human players to predict the behavior patterns of AI through historical games, because the actions of the intelligent agent change according to the real-time environment and dynamic decision-making, and are no longer simple presets or rules.
[0101] 3. Improve user experience: The dynamic strategy of the intelligent agent makes the game more diverse and challenging, improves the overall entertainment and competitiveness of the game, and improves the player's gaming experience.
[0102] 4. Better decision quality: Through deep learning of historical data and strategy optimization, the agent can make better decisions and achieve better performance in the game.
[0103] In general, the method provided can allow the agent to use different outfit styles in different games through the training and use of the prediction model, making the agent more human-like, and allowing the agent to decide the order of upgrading skills and equipment in the game. At the same time, the equipment replacement scheme proposed in this patent allows the agent to have higher strength in the later stages of the game. This enables the game agent to better adapt to the real-time environment, form flexible and changeable strategies, and bring a more challenging and interesting game experience.
[0104] See also Figure 5 As shown, Figure 5 2 is a schematic diagram of the structure of the intelligent agent using device 200 provided in an embodiment of the present application. The intelligent agent using device 200 is used to execute the steps of the intelligent agent using method shown in the above embodiments. The intelligent agent using device 200 can be a single server or a server cluster, or the intelligent agent using device 200 can be a terminal, which can be a handheld terminal, a laptop computer, a wearable device or a robot, etc.
[0105] like Figure 5 As shown, the agent using device 200 includes:
[0106] An information acquisition unit 201 is used to acquire a plurality of historical operation data corresponding to a preset game environment and historical gold coin information corresponding to each historical operation data;
[0107] The operation acquisition unit 202 is used to acquire the historical operation corresponding to each game data; wherein the historical operation includes any one of equipment purchase operation, equipment sale operation, skill upgrade operation and other operations;
[0108] The prediction output unit 203 is used to input a plurality of historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained, and the operation prediction model outputs the predicted operation;
[0109] A training completion unit 204 is used to complete the training of the operation prediction model according to the predicted operation and the corresponding historical operation, and obtain the trained operation prediction model;
[0110] The current acquisition unit 205 is used to acquire the current equipment information, current skill information and current gold coin information corresponding to the agent to be used in the game environment;
[0111] The usage output unit 206 is used to input the current equipment information, current skill information and current gold coin information into the prediction operation model, and the prediction operation model outputs the usage information corresponding to the agent; wherein the usage information includes one or more of equipment purchase operation, equipment sale operation, skill upgrade operation and other operations;
[0112] The usage completion unit 207 is used to complete the usage of the agent in the game environment according to the usage information.
[0113] In some embodiments, the historical gold coin information includes the first historical gold coin of the game character corresponding to the historical operation data; before obtaining the historical operation corresponding to each game data, it also includes: an information acquisition unit, used to obtain equipment information corresponding to the game environment; the equipment information includes multiple equipment and the equipment price corresponding to each equipment; obtaining the minimum equipment price among multiple equipment prices; if the first historical gold coin corresponding to the historical operation data is less than the minimum equipment price, determining that the historical operation data is redundant operation data; removing redundant operation data from multiple historical operation data.
[0114] In some embodiments, before multiple historical operation data and historical gold coin information corresponding to each historical operation data are input into the operation prediction model to be trained, it also includes: a sample division unit, which is used to divide each historical operation data into positive sample data and negative sample data according to the historical operation, wherein the historical operation corresponding to the positive sample data includes any one of equipment purchase operation, equipment sale operation and skill upgrade operation, and the historical operation corresponding to the negative sample data includes other operations; the positive sample data, the historical gold coin information corresponding to the positive sample data and the negative sample data, the historical gold coin information corresponding to the negative sample data are respectively input into the operation prediction model.
[0115] Exemplarily, before respectively inputting the positive sample data, the historical gold coin information corresponding to the positive sample data, and the negative sample data, the historical gold coin information corresponding to the negative sample data into the operation prediction model, it also includes: a quantity adjustment unit, used to adjust the quantity of negative sample data so that the ratio of the quantity of negative sample data to the quantity of positive sample data is a preset ratio.
[0116] In some embodiments, before multiple historical operation data and historical gold coin information corresponding to each historical operation data are input into the operation prediction model to be trained, it also includes: a data copying unit, used to determine the historical operation data of the equipment selling operation as the target data; copy the target data according to a preset number of times to generate multiple backup data; wherein the historical gold coin information corresponding to the backup data is the same as the target data, so as to input multiple backup data, the historical gold coin information corresponding to each backup data, multiple historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained.
[0117] In some embodiments, the historical gold coin information includes the first historical gold coins of the game character corresponding to the historical operation data and the second historical gold coins of the game character's opposing character in the game environment; the prediction output unit 203 is also used to: obtain the first combat power information of the game character corresponding to each historical operation data and the second combat power information of the opposing character; input the historical operation data and the first historical gold coins, the second historical gold coins, the first combat power information and the second combat power information corresponding to each historical operation data into the operation prediction model to be trained.
[0118] In some embodiments, after the use of the intelligent agent in the game environment is completed according to the usage information, it also includes: a model optimization unit, which is used to obtain state information of the intelligent agent and the opposing character corresponding to the intelligent agent in the game environment; obtain difficulty information corresponding to the game environment; and optimize the operation prediction model according to the state information and difficulty information.
[0119] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working processes of the intelligent agent usage device and each module described above can refer to the corresponding processes in the intelligent agent usage method embodiments described in the above embodiments, and will not be repeated here.
[0120] The above-mentioned agent use method can be implemented in the form of a computer program. The computer program can be used in Figure 5 Run on the device shown.
[0121] See also Figure 6 , Figure 6 1 is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device includes a processor, a memory and a network interface connected via a device bus, wherein the memory may include a storage medium and an internal memory.
[0122] The storage medium can store an operating device and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the intelligent agent use methods.
[0123] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.
[0124] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any method of using the intelligent agent.
[0125] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal to which the scheme of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0126] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0127] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:
[0128] Obtain multiple historical operation data corresponding to a preset game environment and historical gold coin information corresponding to each historical operation data;
[0129] Obtaining the historical operations corresponding to each game data; wherein the historical operations include equipment purchase operations, equipment sale operations, skill upgrade operations, and any one of other operations;
[0130] Inputting a plurality of historical operation data and historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained, and the operation prediction model outputs a predicted operation;
[0131] The operation prediction model is trained according to the predicted operation and the corresponding historical operation, and the trained operation prediction model is obtained;
[0132] In the game environment, obtain the current equipment information, current skill information and current gold coin information corresponding to the intelligent agent to be used;
[0133] Input the current equipment information, current skill information and current gold coin information into the prediction operation model, and the prediction operation model outputs the usage information corresponding to the agent; wherein the usage information includes one or more of equipment purchase operation, equipment sale operation, skill upgrade operation and other operations;
[0134] Complete the use of the agent in the game environment based on the usage information.
[0135] In some embodiments, the historical gold coin information includes the first historical gold coin of the game character corresponding to the historical operation data; before obtaining the historical operation corresponding to each game data, it also includes: obtaining equipment information corresponding to the game environment; the equipment information includes multiple equipment and the equipment price corresponding to each equipment; obtaining the minimum equipment price among multiple equipment prices; if the first historical gold coin corresponding to the historical operation data is less than the minimum equipment price, determining that the historical operation data is redundant operation data; removing redundant operation data from multiple historical operation data.
[0136] In some embodiments, before inputting multiple historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained, it also includes: dividing each historical operation data into positive sample data and negative sample data according to the historical operation, wherein the historical operation corresponding to the positive sample data includes any one of equipment purchase operation, equipment sale operation and skill upgrade operation, and the historical operation corresponding to the negative sample data includes other operations; respectively inputting the positive sample data, the historical gold coin information corresponding to the positive sample data and the negative sample data, the historical gold coin information corresponding to the negative sample data into the operation prediction model.
[0137] Exemplarily, before respectively inputting the positive sample data, the historical gold coin information corresponding to the positive sample data, and the negative sample data, the historical gold coin information corresponding to the negative sample data into the operation prediction model, it also includes: adjusting the number of negative sample data so that the ratio of the number of negative sample data to the number of positive sample data is a preset ratio.
[0138] In some embodiments, before inputting multiple historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained, it also includes: determining the historical operation data of the equipment selling operation as the target data; copying the target data according to a preset number of times to generate multiple backup data; wherein the historical gold coin information corresponding to the backup data is the same as the target data, so as to input multiple backup data, the historical gold coin information corresponding to each backup data, multiple historical operation data and the historical gold coin information corresponding to each historical operation data into the operation prediction model to be trained.
[0139] In some embodiments, the historical gold coin information includes the first historical gold coins of the game character corresponding to the historical operation data and the second historical gold coins of the game character and the opposing character in the game environment; multiple historical operation data and the historical gold coin information corresponding to each historical operation data are input into the operation prediction model to be trained, including: obtaining the first combat power information of the game character and the second combat power information of the opposing character corresponding to each historical operation data; inputting the historical operation data and the first historical gold coins, the second historical gold coins, the first combat power information and the second combat power information corresponding to each historical operation data into the operation prediction model to be trained.
[0140] In some embodiments, after completing the use of the intelligent agent in the game environment according to the usage information, it also includes: obtaining state information of the intelligent agent and the opposing character corresponding to the intelligent agent in the game environment; obtaining difficulty information corresponding to the game environment; and completing the optimization of the operation prediction model according to the state information and difficulty information.
[0141] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement the steps of the intelligent agent usage method provided in the above embodiments of the present application.
[0142] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.
[0143] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A method for using an intelligent agent, characterized in that: The method comprises: Acquire multiple historical operation data corresponding to a preset game environment and historical gold coin information corresponding to each of the historical operation data; Acquire the historical operation corresponding to each of the game data; wherein the historical operation includes any one of an equipment purchase operation, an equipment sale operation, a skill upgrade operation, and other operations; Inputting the plurality of historical operation data and the historical gold coin information corresponding to each of the historical operation data into the operation prediction model to be trained, and the operation prediction model outputs the predicted operation; Complete the training of the operation prediction model according to the predicted operation and the corresponding historical operation, and obtain the trained operation prediction model; In the game environment, obtaining current equipment information, current skill information, and current gold coin information corresponding to the intelligent agent to be used; The current equipment information, current skill information and current gold coin information are input into the prediction operation model, and the prediction operation model outputs usage information corresponding to the agent; wherein the usage information includes one or more of equipment purchase operation, equipment sale operation, skill upgrade operation and other operations; The use of the agent is completed in the game environment according to the use information.
2. The method according to claim 1, characterized in that The historical gold coin information includes the first historical gold coin of the game character corresponding to the historical operation data; before obtaining the historical operation corresponding to each of the game data, it also includes: Acquire equipment information corresponding to the game environment; the equipment information includes a plurality of equipment and an equipment price corresponding to each of the equipment; Obtaining a minimum equipment price among a plurality of equipment prices; If the first historical gold coin corresponding to the historical operation data is less than the minimum equipment price, determining that the historical operation data is redundant operation data; The redundant operation data is removed from the plurality of historical operation data.
3. The method according to claim 1, characterized in that Before inputting the plurality of historical operation data and the historical gold coin information corresponding to each of the historical operation data into the operation prediction model to be trained, the method further includes: Dividing each of the historical operation data into positive sample data and negative sample data according to the historical operation, wherein the historical operation corresponding to the positive sample data includes any one of an equipment purchase operation, an equipment sale operation, and a skill upgrade operation, and the historical operation corresponding to the negative sample data includes the other operations; The positive sample data, the historical gold coin information corresponding to the positive sample data, and the negative sample data, the historical gold coin information corresponding to the negative sample data are respectively input into the operation prediction model.
4. The method according to claim 3, characterized in that Before respectively inputting the positive sample data and the historical gold coin information corresponding to the positive sample data and the negative sample data and the historical gold coin information corresponding to the negative sample data into the operation prediction model, the method further includes: The amount of the negative sample data is adjusted so that a ratio of the amount of the negative sample data to the amount of the positive sample data is a preset ratio.
5. The method according to claim 1, characterized in that: Before inputting the plurality of historical operation data and the historical gold coin information corresponding to each of the historical operation data into the operation prediction model to be trained, the method further includes: determining the historical operation data in which the historical operation is the equipment selling operation as target data; The target data is copied according to a preset number of times to generate multiple backup data; wherein the historical gold coin information corresponding to the backup data is the same as the target data, so that the multiple backup data, the historical gold coin information corresponding to each backup data, the multiple historical operation data and the historical gold coin information corresponding to each historical operation data are input into the operation prediction model to be trained.
6. The method according to claim 1, characterized in that The historical gold coin information includes the first historical gold coin of the game character corresponding to the historical operation data and the second historical gold coin of the game character's opponent in the game environment; the inputting of the plurality of historical operation data and the historical gold coin information corresponding to each of the historical operation data into the operation prediction model to be trained comprises: Obtaining first combat power information of the game character and second combat power information of the opposing character corresponding to each of the historical operation data; The historical operation data and the first historical gold coin, the second historical gold coin, the first combat power information and the second combat power information corresponding to each of the historical operation data are input into the operation prediction model to be trained.
7. The method according to claim 1, characterized in that After completing the use of the agent in the game environment according to the use information, the method further includes: Acquire status information of the agent and the opposing character corresponding to the agent in the game environment; Obtaining difficulty information corresponding to the game environment; The operation prediction model is optimized according to the state information and the difficulty information.
8. An intelligent agent using device, characterized in that: include: An information acquisition unit, used to acquire a plurality of historical operation data corresponding to a preset game environment and historical gold coin information corresponding to each of the historical operation data; An operation acquisition unit, used to acquire the historical operation corresponding to each of the game data; wherein the historical operation includes any one of an equipment purchase operation, an equipment sale operation, a skill upgrade operation and other operations; A prediction output unit, used for inputting the plurality of historical operation data and the historical gold coin information corresponding to each of the historical operation data into the operation prediction model to be trained, and the operation prediction model outputs the predicted operation; A training completion unit, used to complete the training of the operation prediction model according to the predicted operation and the corresponding historical operation, and obtain the trained operation prediction model; A current acquisition unit, used to acquire current equipment information, current skill information and current gold coin information corresponding to the intelligent agent to be used in the game environment; A usage output unit is used to input the current equipment information, current skill information and current gold coin information into the prediction operation model, and the prediction operation model outputs usage information corresponding to the agent; wherein the usage information includes one or more of equipment purchase operation, equipment sale operation, skill upgrade operation and other operations; A usage completion unit is used to complete the use of the agent in the game environment according to the usage information.
9. A computer device, characterized in that: The computer device comprises: Memory and processor; Wherein, the memory is connected to the processor and is used to store programs; The processor is configured to implement the steps of the method according to any one of claims 1 to 7 by running the program stored in the memory.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the steps of the method according to any one of claims 1 to 7.