Task triggering test method and terminal based on multi-agent reinforcement learning
Through multi-agent reinforcement learning, explorer, collaborator and distractor characters generated, combined with the Actor-Critic model and reward function, the problem of low task trigger coverage in game testing is solved, and efficient and accurate task trigger testing is achieved in complex environments.
Patent Information
- Application Number
- CN202510573935.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-06
- Publication Date
- 2025-08-01
AI Technical Summary
In the prior art, game testing, especially task trigger verification of open world or multi-task chain games, relies on manual testing or single behavioral automatic testing, which is difficult to cover all trigger paths, and lacks simulation of complex environments and multi-character collaboration, making it difficult for the test results to reflect the actual gaming experience.
Using a multi-agent reinforcement learning method, multiple agent roles are generated, including explorers, collaborators and distractors, dynamic course hierarchical training is used to maximize task trigger conditions coverage, use the Actor-Critic model to optimize agent decisions, and guide behavior through reward functions to generate test cases to verify task logic and progress.
It achieves comprehensive coverage of task trigger conditions in complex environments, ensures the accuracy of task scenarios and chain conditions, and improves the authenticity and efficiency of tests, especially in dynamic task chains and changing environments, which can adaptively optimize action decisions.
Smart Images
Figure CN120407421A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of game testing, and particularly to a task trigger testing method and a terminal based on multi-agent reinforcement learning. Background Art
[0002] In the prior art, game testing, especially the task trigger verification of open-world or multi-task chain games, still mainly relies on manual testing or single-behavior automated testing based on scripts. These methods generally have the following problems: First, the task trigger conditions are complex and there are hidden conditions, and it is difficult for manual testing to exhaustively cover all trigger paths, resulting in potential logic errors being missed; Second, the execution scope of traditional automated test scripts is limited, and they can often only cover preset paths and cannot handle complex situations such as dynamic interactions, environmental disturbances, or multi-role collaborations; Third, existing test systems usually mainly execute operations with a single agent, lacking the simulation of real interactive behaviors such as cooperation and resource competition among players, and the test results are difficult to reflect the actual game experience. Summary of the Invention
[0003] The technical problem to be solved by the present invention is: to provide a task trigger testing method and a terminal based on multi-agent reinforcement learning to solve the problem of low coverage rate of test task trigger situations.
[0004] To solve the above technical problem, the technical solution adopted by the present invention is: A task trigger testing method based on multi-agent reinforcement learning, comprising the steps of: Generating multiple agent roles based on a reinforcement learning algorithm and controlling the agent roles to execute actions; the agent roles include at least one explorer, at least one collaborator, and at least one disruptor; the explorer is used to trigger task conditions, the collaborator is used to execute task collaboration actions, and the disruptor is used to simulate role competition behaviors; The agent roles are trained using dynamic curriculum grading and aim to maximize the coverage rate of task trigger conditions; Generating test cases according to the training results, and the test cases are used to verify the correctness of the trigger logic and task progress of game tasks.
[0005] To solve the above technical problem, another technical solution adopted by the present invention is: A task trigger testing terminal based on multi-agent reinforcement learning, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are completed: Based on the reinforcement learning algorithm, multiple agent roles are generated and the agent roles are controlled to perform actions; the agent roles include at least one explorer, at least one collaborator, and at least one disruptor; the explorer is used to trigger task conditions, the collaborator is used to perform task collaboration actions, and the disruptor is used to simulate role competition behavior; The agent roles are trained using dynamic curriculum grading with the goal of maximizing the coverage rate of task trigger conditions; Test cases are generated according to the training results, and the test cases are used to verify the correctness of the trigger logic and task progress of the game task.
[0006] The beneficial effects of the present invention are as follows: A task trigger test method and terminal based on multi-agent reinforcement learning are provided. By generating multiple agent roles and controlling their actions, multi-agent reinforcement learning is used to simulate the trigger conditions of game tasks; in the above process, through the cooperation and confrontation of multiple agents, the trigger conditions of game tasks can be comprehensively tested in a complex environment to ensure that all task scenarios and task chain conditions are covered. At the same time, the agents are trained through dynamic curriculum grading, enabling them to adaptively optimize action decisions, not only covering more task trigger scenarios but also accurately executing tasks. Especially in dynamic task chains and changing environmental conditions, the accuracy of task triggering can be guaranteed. Description of the Drawings
[0007] Figure 1 It is a flowchart of a task trigger test method based on multi-agent reinforcement learning in an embodiment of the present invention; Figure 2 It is a schematic diagram of a task trigger test terminal based on multi-agent reinforcement learning in an embodiment of the present invention; Label Description: 1. A task trigger test terminal based on multi-agent reinforcement learning; 2. Memory; 3. Processor. Detailed Embodiment
[0008] To describe the technical content, achieved objectives, and effects of the present invention in detail, the following is described in conjunction with the embodiments and with reference to the drawings.
[0009] Please refer to Figure 1 , a task trigger test method based on multi-agent reinforcement learning, includes the steps: Based on the reinforcement learning algorithm, multiple agent roles are generated and the agent roles are controlled to perform actions; the agent roles include at least one explorer, at least one collaborator, and at least one disruptor; the explorer is used to trigger task conditions, the collaborator is used to perform task collaboration actions, and the disruptor is used to simulate role competition behavior; The intelligent agent roles adopt dynamic curriculum grading training and aim to maximize the coverage rate of task triggering conditions; Generate test cases based on the training results, and the test cases are used to verify the triggering logic of game tasks and the correctness of task progress.
[0010] It can be understood that in multi-agent reinforcement learning, the roles of explorers, collaborators, and disruptors each undertake different tasks to achieve comprehensive coverage of task triggering conditions. Specifically, explorers promote the progress of tasks by triggering specific task conditions, collaborators focus on providing support during task execution, such as working together or enhancing the capabilities of other agents, and disruptors simulate competition among roles to test the robustness of the system and the integrity of task triggering conditions by interfering with the task process.
[0011] In this process, the goal of maximizing the coverage rate of task triggering conditions is to ensure that all possible task triggering conditions can be verified in different task scenarios through the collaboration and competition of these roles. This includes not only the known task chain conditions but also potential triggering conditions that have not been directly demonstrated, such as hidden conditions of certain tasks or tasks that are triggered only in special scenarios.
[0012] The interaction inputs and behaviors of these intelligent agent roles not only depend on the reinforcement learning process of the model but also need to make real-time responses to the dynamically changing task conditions in the environment, so as to maximize the comprehensive coverage of task triggering conditions.
[0013] As can be seen from the above description, the beneficial effects of the present invention are as follows: By generating multiple intelligent agent roles and controlling their execution actions, multi-agent reinforcement learning is used to simulate the triggering conditions of game tasks. Each intelligent agent role (explorer, collaborator, and disruptor) has different goals during task execution. The explorer triggers task conditions, the collaborator assists in task completion, and the disruptor simulates competitive behaviors. Through this method, it is possible to systematically verify whether task triggering conditions are fully covered in various scenarios, ensuring the accuracy of task triggering logic and task progress.
[0014] At the same time, through the cooperation and confrontation of multiple agents, it is possible to comprehensively test the triggering conditions of game tasks in a complex environment, ensuring that all task scenarios and task chain conditions are covered. Compared with traditional single-agent testing methods, this method can simulate a more realistic game environment.
[0015] Moreover, by training the agents to be able to adaptively optimize action decisions, it can not only cover more task triggering scenarios but also accurately execute tasks. Especially in dynamic task chains and changing environmental conditions, it can ensure the accuracy of task triggering.
[0016] Specifically, a Hash structure is used to store the agent state (Key: AgentID, Field: Timestamp), supporting high-concurrency read and write (60Hz synchronization).
[0017] In some embodiments, the agent role is generated based on a reinforcement learning algorithm, the reinforcement learning algorithm includes an Actor-Critic model, and the Actor-Critic model includes an Actor neural network and a Critic neural network; the Actor neural network is used to generate the action decision of the agent role; the Critic network evaluates the global value of the action decision and adjusts the policy of the Actor network by calculating the error.
[0018] As can be seen from the above description, the Actor-Critic model is used to optimize the action decision of the agent. The Actor network is responsible for generating the actions of each agent, while the Critic network evaluates the overall value of these actions and helps the Actor adjust its policy by calculating the error. This method improves the cooperation efficiency of the multi-agent system and the coverage of task triggering through the dual feedback mechanism of Actor-Critic. Through the Actor-Critic model, the agent can not only optimize its behavior by itself (generating decisions through the Actor network), but also evaluate the global effects of these decisions through the Critic network and optimize the coverage of task triggering conditions. The Critic network calculates the error and feeds it back to the Actor, enabling the model to continuously adjust the policy and improve the cooperation and efficiency among multiple agents. Especially when facing a dynamic environment and complex task chains, the agent can quickly adapt and optimize the execution of tasks.
[0019] For example, in game testing, assume a task involves multiple steps. For example, the explorer needs to trigger a mechanism, the collaborator needs to heal or assist, and the disruptor needs to interrupt the task. The Actor network generates the action decisions of the explorer and the collaborator, while the Critic network evaluates the effects based on the results of these decisions (such as whether the task is successfully triggered, whether the task is completed on time, etc.). Through continuous learning and optimization, the agent can gradually improve the triggering effect of the task.
[0020] Specifically, the Actor-Critic model also includes a reward function, and the reward function is used to guide the Actor neural network to optimize the action decision it generates. The formula of the reward function is as follows: R = 0.5×ΔTnew + 0.3×Synergy - 0.2×Conflict; In the formula, R represents the reward function; ΔTnew represents the number of newly triggered tasks of the agent role; Synergy represents the collaboration score among the agent roles; Conflict represents the number of competition times of the agent roles.
[0021] As can be seen from the above description, the reward function is designed to evaluate the performance of the agent in the task and guide the optimization of the Actor network through this function. The reward function comprehensively considers the newly increased number of task trigger conditions (ΔTnew), the collaboration effect among agents (Synergy), and the number of competition behaviors (Conflict). This design helps to promote collaboration among agents while avoiding excessive competition. Through the guidance of the reward function, the agent can better understand which behaviors are beneficial (such as increasing the number of task triggers and improving collaboration efficiency), and which behaviors are harmful (such as frequent competition leading to task failure), so as to accurately optimize its decision-making process. The Synergy part in the reward function encourages agents to collaborate, while the Conflict part punishes excessive competition, prompting agents to cooperate more to improve the overall task completion efficiency.
[0022] For example, assume that in a cooperative task, after the explorer triggers a task, the collaborator needs to heal and cooperate with the explorer to jointly promote the task, while the disruptor will interfere with the task progress. The reward function gives rewards according to the collaboration degree (Synergy) between the explorer and the collaborator, and punishes according to the competition behavior (Conflict) of the disruptor. By adjusting the reward function, the behavior of the agent is gradually optimized, and finally the task is efficiently completed.
[0023] Preferably, the Actor neural network includes six fully connected layers, corresponding to the individual decision-making units in the multi-agent system, responsible for generating personalized action strategies for each agent (explorer / collaborator / disruptor). Its input and output process is as follows: Input 512 dimensions: Integrate the agent's own state (coordinates, health, skill cooldown) and environmental observations (coverage heat map) Hidden layer 1024×3: Realize multi-task strategy generation through deep feature extraction (such as the explorer activating the mechanism and the disruptor snatching resources) Output 256 dimensions: Map to the action space (8-direction movement × 3 skills × 3 interactions), supporting compound behavior decision-making.
[0024] The Critic neural network contains five fully connected layers. As a global coordinator, it evaluates the overall value of multi-agent collaboration and solves the conflict problem between local optimality and global goals. Its input and output process is as follows: Input 512 dimensions: Integrate the states of all agents (Redis cluster synchronizes data) and the task progress Output a 1D scalar: Quantify the current collaboration efficiency (such as Synergy score, conflict penalty) to guide the update of the Actor network policy.
[0025] In some embodiments, the agent role adopts dynamic curriculum grading training, including: Obtain the actions of the agent role and generate task chain data and the dynamic event types in the task chain data based on the action situation; Based on the task chain data, the dynamic event types, and the proportion of disruptors in the agent role, perform complexity calculation; According to the complexity calculation result, match the training level according to the gradient, and perform training according to the matched training level.
[0026] It can be understood that task chain data and dynamic event types are two key factors used to quantify task complexity and environmental dynamics. They play an important role in the complexity calculation formula and directly affect the training process and the design of task trigger tests in multi-agent reinforcement learning.
[0027] Task chain data refers to the dependencies between tasks and the number of tasks in the game tasks. In an open-world game, tasks are usually composed of multiple subtasks, and the subtasks can form a "chain" through front-back associations, that is, the completion of one task may be a condition for the start of another task. For example, in a game dungeon, after completing the first task, the second task may be triggered, and only after the second task is completed, the third task will be started. These task dependencies constitute the task chain.
[0028] Dynamic event types refer to various dynamic events that may occur in the game, such as weather changes, environmental factors (such as day-night alternation), emergencies (such as the appearance of random enemies), etc. These factors will affect the progress of the game and the task trigger conditions. Its value reflects the changing factors in the game environment that may affect the task progress. The more types and quantities of dynamic events, the more variables and situations need to be considered in the test, thus increasing the difficulty and complexity of training. For example, in some game tasks, weather changes may affect the combat ability of the character, or in a certain task, the player needs to make different decisions according to the weather conditions.
[0029] From the above description, it can be seen that the training process obtains the actions of the agent role and generates task chain data and dynamic event types based on the action situation, and then performs complexity calculation. The complexity calculation combines task chain data, dynamic event types, and the proportion of disruptors to dynamically match the training level. By matching the training level according to the gradient, the system can dynamically adjust the training difficulty according to the complexity of the task, thereby improving the learning efficiency of the agent.
[0030] Specifically, the complexity calculation formula is as follows: C = 0.5N task + 0.3N event + 0.2N agent ; In the formula, C represents complexity; N task represents the task chain data; N event represents the dynamic event type; N agent represents the proportion of the disruptor in the agent role.
[0031] Preferably, each of the training levels represents: a) Low level: Applicable to test scenarios with simple task chains and a small number of dynamic events, mainly training the basic behaviors of the agent, such as task condition triggering and basic collaboration behaviors; this level requires a high coverage rate (≥95%) to ensure the integrity of basic tasks; b) Middle level: Applicable to complex scenarios with nested task chains and a small number of disruptors, introducing multiple task combinations and some dynamic events (such as weather changes or the appearance of enemies), training the agent's collaboration and complex event response capabilities; the training objective of this level is to maintain a relatively high coverage rate and interaction fidelity within a reasonable range; c) High level: Applicable to scenarios with complex task chains, dynamic events, and multiple disruptors, introducing comprehensive task chain nesting and complex environmental changes (such as the impact of heavy rain weather on task progress), training the agent's collaboration and decision-making capabilities in a multi-task and high-interference environment; this level allows reducing the coverage rate requirement for task triggering (such as 85%) to improve efficiency.
[0032] In addition, when loading the model parameters of the lower training level during training at a higher training level (such as loading the weights of the Actor network of the lower level at the middle level), the first 2 layers of the network are frozen to retain the basic strategy, preventing catastrophic forgetting, and allowing the last 3 layers (policy optimization layers) of the higher level network to be fine-tuned to adapt to complex scenarios.
[0033] In some embodiments, the test cases are generated from the key event sequences extracted from the agent's behavior trajectories, and the key event sequences at least include task triggering, role skill release, and role movement behaviors. As can be seen from the above description, test cases are generated through the key event sequences in the agent's behavior trajectory. Key events include task triggering, character skill release, and character movement, etc. By extracting these key events, the task execution process can be accurately reproduced, effective test cases can be generated and used to verify the task logic and task progress. By automatically extracting key events and generating test cases, it can ensure that the logic and progress of the task are comprehensively verified, avoid missing important test scenarios, and improve test efficiency. All key information is contained in the agent's behavior trajectory, and the process of task triggering and execution can be accurately reproduced through script generation, thus ensuring the accuracy and repeatability of the test.
[0034] Please refer to Figure 2 , a task triggering test terminal 1 based on multi-agent reinforcement learning, includes a memory 2, a processor 3, and a computer program stored on the memory 2 and executable on the processor 3. When the processor executes the computer program, it completes the steps in a task triggering test method based on multi-agent reinforcement learning.
[0035] In summary, the present invention realizes high-coverage verification of the game task triggering logic by constructing a task triggering test method based on multi-agent reinforcement learning and its terminal, and improves the test accuracy and efficiency in complex environments. By generating multiple agent characters and assigning different behavior goals, the explorer triggers the task conditions, the collaborator completes the task collaboration, and the disruptor creates competitive perturbations, simulating various player behavior patterns. In the training, the Actor-Critic architecture is combined for strategy optimization. The Critic evaluates the strategy value to guide the Actor to iterate the action selection. At the same time, a complexity calculation formula based on task chain data, dynamic event types, and the proportion of disruptors is introduced to match different training levels according to the gradient, promoting the agent to gradually adapt to the task scenarios from simple to complex. The reward function comprehensively considers the number of task triggers, collaboration scores, and the number of conflicts, guiding the agent to tend to an effective strategy with high collaboration and low interference. Through the structured extraction of trajectory data during the training process, test cases containing key events are automatically generated, effectively covering the main and branch lines, hidden conditions, and variable environmental factors in the task chain. The system can not only dynamically adapt the training strategy, but also has scalability and high-concurrency state management capabilities, significantly improving the performance of task triggering tests in terms of authenticity, comprehensiveness, and efficiency.
[0036] The above are only the embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in the related technical fields, shall be equally included in the patent protection scope of the present invention.
Claims
1. A task trigger test method based on multi-agent reinforcement learning, characterized in that: Including the steps: Based on the reinforcement learning algorithm, generate multiple agent roles and control the agent roles to perform actions; the agent roles include at least one explorer, at least one collaborator, and at least one disruptor; the explorer is used to trigger task conditions, the collaborator is used to perform task collaboration actions, and the disruptor is used to simulate role competition behaviors; The agent roles are trained using dynamic curriculum grading and aim to maximize the coverage rate of task trigger conditions; Generate test cases according to the training results, and the test cases are used to verify the triggering logic of the game task and the correctness of the task progress.
2. The task trigger test method based on multi-agent reinforcement learning according to claim 1, wherein: The agent roles are generated based on the reinforcement learning algorithm, and the reinforcement learning algorithm includes an Actor-Critic model. The Actor-Critic model includes an Actor neural network and a Critic neural network; the Actor neural network is used to generate action decisions for the agent roles; the Critic network evaluates the global value of the action decisions and adjusts the strategy of the Actor network by calculating errors.
3. The task trigger test method based on multi-agent reinforcement learning according to claim 2, characterized in that: The Actor-Critic model further includes a reward function, and the reward function is used to guide the Actor neural network to optimize the action decisions it generates. The formula of the reward function is expressed as follows: R = 0.5×ΔTnew + 0.3×Synergy - 0.2×Conflict; In the formula, R represents the reward function; ΔTnew represents the number of newly triggered tasks of the agent roles; Synergy represents the collaboration score among the agent roles; Conflict represents the number of competition times of the agent roles.
4. The task triggering test method based on multi-agent reinforcement learning according to claim 1, characterized in that: The agent roles are trained using dynamic curriculum grading, including: Obtain the actions of the agent roles and generate task chain data and the dynamic event types in the task chain data based on the action situations; Perform complexity calculation based on the task chain data, the dynamic event types, and the proportion of the disruptor in the agent roles; Match the training levels according to the gradient based on the complexity calculation results, and perform training according to the matched training levels.
5. The task trigger test method based on multi-agent reinforcement learning according to claim 1, characterized in that: The test cases are generated from the key event sequences extracted from the agent behavior trajectories, and the key event sequences at least include task triggers, role skill releases, and role movement behaviors.
6. A task trigger test terminal based on multi-agent reinforcement learning, characterized in that: Including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are completed: Based on the reinforcement learning algorithm, generate multiple agent roles and control the agent roles to perform actions; the agent roles include at least one explorer, at least one collaborator, and at least one disruptor; the explorer is used to trigger task conditions, the collaborator is used to perform task collaboration actions, and the disruptor is used to simulate role competition behaviors; The agent roles are trained using dynamic curriculum grading and aim to maximize the coverage rate of task trigger conditions; Generate test cases according to the training results, and the test cases are used to verify the triggering logic of game tasks and the correctness of task progress.
7. The task trigger test terminal based on multi-agent reinforcement learning according to claim 6, characterized in that: The intelligent agent role is generated based on a reinforcement learning algorithm, and the reinforcement learning algorithm includes an Actor-Critic model. The Actor-Critic model includes an Actor neural network and a Critic neural network. The Actor neural network is used to generate action decisions for the intelligent agent role. The Critic network evaluates the global value of the action decisions and adjusts the policy of the Actor network by calculating the error.
8. The task trigger test terminal based on multi-agent reinforcement learning according to claim 7, characterized in that: The Actor-Critic model further includes a reward function, and the reward function is used to guide the Actor neural network to optimize the action decisions it generates. The formula of the reward function is expressed as follows: R = 0.5×ΔTnew + 0.3×Synergy - 0.2×Conflict; In the formula, R represents the reward function; ΔTnew represents the number of newly triggered tasks of the intelligent agent role; Synergy represents the collaboration score among the intelligent agent roles; Conflict represents the number of competition times of the intelligent agent role.
9. The task trigger test terminal based on multi-agent reinforcement learning according to claim 6, characterized in that: The intelligent agent role adopts dynamic curriculum hierarchical training, including: Obtain the actions of the intelligent agent role and generate task chain data and the dynamic event types in the task chain data based on the action situation; Based on the task chain data, the dynamic event types, and the proportion of the disruptor in the intelligent agent role, perform complexity calculation; According to the complexity calculation result, match the training level according to the gradient, and perform training according to the matched training level.
10. The task trigger test terminal based on multi-agent reinforcement learning according to claim 6, characterized in that: The test cases are generated from the key event sequences extracted from the intelligent agent behavior trajectories, and the key event sequences at least include task triggering, character skill release, and character movement behaviors.