An agent control method, apparatus and device

Abstract

The application provides an agent control method, device and equipment to solve the problem of poor agent performance and poor player experience. The method comprises: in response to an agent adding operation triggered for a target game, obtaining respective corresponding script tasks of at least one agent to be added; for each game stage of each agent in the target game, respectively performing: based on the scene description information of the stage scene corresponding to one game stage, the role attribute information of one agent in one game stage, and the stage task of the script task of one agent corresponding to one game stage, determining the execution action set of one agent in one game stage; obtaining the game state information of one agent, the game state information comprising: when the current role state and the current acquisition condition both satisfy the execution condition of the execution action set, controlling one agent to execute the execution action set in one game stage.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a control method, apparatus and device for an intelligent agent. Background Technology

[0002] With the increasing popularity of games, most games add game agents (i.e., non-player characters, or NPCs) to interact with real players, which can drive the plot forward, provide real players with an immersive gaming experience, and thus enhance the game's playability and richness.

[0003] To enhance the gaming experience for real players, the design of game agents typically includes behavior tree schemes and reinforcement learning schemes.

[0004] The behavior tree approach refers to decomposing an agent's behavioral decision-making process into multiple simple tasks, each of which can be achieved through different types of nodes in the behavior tree. In the behavior tree, different types of nodes represent a behavior, condition, or decision of the agent. By combining multiple nodes in different hierarchical ways, the agent can execute different behavioral logics.

[0005] However, the combinations of nodes in a behavior tree scheme are predefined. The agent can only execute behavioral decisions according to the combinations of nodes in the behavior tree, leading to behavioral rigidity and, in practical business applications, the agent's behavior may appear unhuman-like. Furthermore, as the agent's behavioral decisions become increasingly complex, more nodes need to be set in the behavior tree, resulting in an explosive growth of nodes, leading to low development efficiency and difficulty in maintenance.

[0006] Reinforcement learning, in essence, uses an intelligent agent model to simulate a real player. A reward mechanism is designed based on the game's script tasks to allocate resources to the agent and thus facilitate the completion of those tasks. However, if this reward mechanism is flawed, it can lead to excessive resource allocation to the real player, resulting in a decreased human-likeness and strength of the agent, ultimately negatively impacting the player's experience.

[0007] For example, in a game, real players typically acquire resources in their lane before supporting other players. However, to ensure real players complete a task, the reward mechanism might require AI agents to continuously support them. Yet, in this support process, AI agents might forgo their own resources, leading to a decrease in their strength. Alternatively, when both an AI agent and a real player are simultaneously acquiring a resource, the AI ​​agent might abandon acquiring that resource to ensure the player completes a task, mirroring the actions of a real player. This results in the AI ​​agent's actions not closely resembling those of a real player, negatively impacting the real player's experience. Summary of the Invention

[0008] This application provides a method, apparatus, and device for controlling an intelligent agent, in order to solve the problems of the intelligent agent's non-human-like performance and poor player experience.

[0009] In a first aspect, embodiments of this application provide a control method for an intelligent agent, including:

[0010] In response to an agent addition operation triggered for the target game, obtain the script tasks corresponding to at least one agent to be added;

[0011] For each agent in each game phase of the target game, perform the following operations:

[0012] Based on the scene description information of the stage scene corresponding to a game stage, the role attribute information of an agent in the game stage, and the stage task corresponding to the script task of the agent in the game stage, the set of actions to be executed by the agent in the game stage is determined.

[0013] Obtain the game state information of the intelligent agent, the game state information including: the current character state of the intelligent agent, and the current acquisition status of the intelligent agent for reward resources;

[0014] When both the current character state and the current acquisition status meet the execution conditions of the action set, the agent is controlled to execute the action set in the game phase.

[0015] Secondly, embodiments of this application provide a control device for an intelligent agent, comprising:

[0016] The acquisition unit is used to acquire the script tasks corresponding to at least one agent to be added in response to an agent addition operation triggered for the target game.

[0017] The processing unit is configured to perform the following operations for each agent in each game phase of the target game:

[0018] Based on the scene description information of the stage scene corresponding to a game stage, the role attribute information of an agent in the game stage, and the stage task corresponding to the script task of the agent in the game stage, the set of actions to be executed by the agent in the game stage is determined.

[0019] The game state information of the intelligent agent is obtained by the acquisition unit. The game state information includes: the current character state of the intelligent agent, and the current acquisition status of the intelligent agent for reward resources.

[0020] When both the current character state and the current acquisition status meet the execution conditions of the action set, the agent is controlled to execute the action set in the game phase.

[0021] In one possible implementation, the processing unit is specifically used for:

[0022] Obtain the preset game stages;

[0023] For a given intelligent agent, determine the behavioral control logic of the intelligent agent in each game phase;

[0024] Based on the preset execution order between the game stages, the behavioral control logic corresponding to each game stage is combined to obtain the script task of the intelligent agent.

[0025] In one possible implementation, the processing unit is specifically used for:

[0026] The script building interface is presented, which displays a judgment node pool and an intervention node pool. Each judgment node represents an execution condition, and each execution condition includes: parameter requirements for triggering the corresponding agent to execute the corresponding set of execution actions. Each intervention node represents an intervention instruction, and each intervention instruction is used to instruct the adjustment and processing of the execution action set corresponding to the corresponding agent.

[0027] In response to the selection operation for the judgment node pool and the intervention node pool, at least one selected target judgment node and at least one target intervention node are presented in the script building interface;

[0028] In response to the assembly operation, the at least one target judgment node and the at least one target intervention node are assembled according to the indicated assembly order to obtain the behavior control logic of the agent in a game phase.

[0029] In one possible implementation, when the processing unit assembles the at least one target judgment node and the at least one target intervention node according to the indicated assembly order in response to the assembly operation, it is specifically configured to:

[0030] In response to a drag operation on the at least one target judgment node, the assembly order among the at least one target judgment node is determined;

[0031] In response to a connection operation between any two adjacent target decision nodes, assembly is performed between the at least one target decision node;

[0032] For each of the target judgment nodes, determine the corresponding target intervention node;

[0033] In response to the connection operation between each target judgment node and its corresponding target intervention node, each target judgment node and its corresponding target intervention node are assembled.

[0034] In one possible implementation, the processing unit is further configured to:

[0035] When the game state information of an agent does not meet the execution conditions of the set of actions, at least one action in the set of actions is adjusted based on the intervention instruction corresponding to the execution conditions.

[0036] In one possible implementation, when the processing unit determines the set of actions to be performed by the intelligent agent in a game stage based on the scene description information of the stage scene corresponding to a game stage, the role attribute information of an agent in the game stage, and the stage task of the script task of the intelligent agent corresponding to the game stage, it is specifically used to:

[0037] Based on the scene description information of the stage scene corresponding to the game stage, the task type corresponding to the stage task is obtained from multiple task types;

[0038] Scene features are extracted from the scene description information to obtain the corresponding scene features;

[0039] The character attribute information is subjected to character feature extraction to obtain the corresponding character attribute features;

[0040] Based on the task type corresponding to the stage task, combined with the scene features and the character attribute features, the set of actions to be performed by the agent in the game stage is predicted.

[0041] In one possible implementation, the scene features and the character attribute features are extracted using a target action prediction model; the target action prediction model includes a feature extraction module and multiple output modules, with different output modules used to output predicted action sets for different task types; the processing unit is further configured to obtain the target action prediction model in the following manner:

[0042] The feature extraction module extracts features from each first sample data of the first task type, and the first output module associated with the first task type outputs the first predicted action set corresponding to each sample feature and the loss value generated based on the first predicted action set. The feature extraction module and the first output module are then tuned. The first task type can be any one of multiple task types.

[0043] For each task type other than the first task type, the following steps are performed: feature extraction is performed on each second sample data of a task type through the parameter-tuned feature extraction module; the second predicted action set corresponding to each sample feature is output through other output modules associated with the task type; and the other output modules are parameter-tuned based on the loss value generated by the second predicted action set.

[0044] In one possible implementation, when the processing unit determines that both the current role state and the current acquisition situation satisfy the execution conditions of the action set, it is specifically used to:

[0045] The behavior control logic of the agent in a game phase is obtained, and the execution conditions in the behavior control logic are determined; wherein, the execution conditions include at least one state execution sub-condition and at least one resource execution sub-condition, each state execution sub-condition representing: the state requirement of a set state that the role state of the agent must meet, and each resource execution sub-condition representing: the resource acquisition situation of the agent must meet a set resource acquisition requirement;

[0046] If the number of state execution sub-conditions satisfied by the current role state reaches a first quantity threshold, and the number of resource execution sub-conditions satisfied by the current acquisition situation reaches a second quantity threshold, it is determined that both the current role state and the current acquisition situation satisfy the execution conditions of the execution action set.

[0047] Thirdly, embodiments of this application provide an electronic device, including:

[0048] Memory, used to store program instructions;

[0049] A processor is configured to invoke program instructions stored in the memory and execute the methods described in the first aspect and in different implementations according to the obtained program instructions.

[0050] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the methods described in the first aspect and in different implementations.

[0051] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the methods described in the first aspect and in different implementations.

[0052] The beneficial effects of this application are as follows:

[0053] After determining the action set of the intelligent agent, this application determines whether the agent meets the execution conditions based on the current character state and the current acquisition status of reward resources. If the execution conditions are met, the corresponding action set is executed; otherwise, at least one action in the action set is adjusted according to the intervention instruction corresponding to the execution conditions. Through this method, the intelligent agent can ensure its acquisition of game resources during gameplay, thereby increasing its strength, making its performance more similar to that of a real player, and enhancing the gaming experience for real players.

[0054] Secondly, in constructing the script task, this application obtains the script by selecting and assembling decision nodes and intervention nodes on the script building page. The decision nodes and intervention nodes in the script building interface are configurable and draggable, improving node reusability and script building efficiency.

[0055] Furthermore, this application predicts the actions of an agent based on an action prediction model, which includes multiple output modules that predict action sets for different task types. During training, if a new task type is added, only the corresponding output module needs to be added to the output layer of the action prediction model, and the parameters of the corresponding output module need to be tuned. There is no need to adjust other network parameters in the action prediction model, which can significantly reduce the training cost of the model.

[0056] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0059] Figure 2 A flowchart of a control method for an intelligent agent provided in an embodiment of this application;

[0060] Figure 3 This is a schematic diagram of an interface for loading an intelligent agent, provided in an embodiment of this application.

[0061] Figure 4 This is a schematic diagram illustrating a method for determining a task type provided in an embodiment of this application;

[0062] Figure 5 This application provides a schematic diagram of a scenario corresponding to different task types in an embodiment of the present application;

[0063] Figure 6 A schematic diagram illustrating a scenario provided in an embodiment of this application;

[0064] Figure 7 A schematic diagram of a game interface provided in this application embodiment;

[0065] Figure 8 A schematic diagram of role levels provided in this application embodiment;

[0066] Figure 9 A flowchart of an adjustment execution action set provided in an embodiment of this application;

[0067] Figure 10 A schematic diagram illustrating another set of adjustment execution actions provided in an embodiment of this application;

[0068] Figure 11 This is a schematic diagram of another set of adjustment execution actions provided in an embodiment of this application;

[0069] Figure 12 A schematic diagram of a script building interface provided in an embodiment of this application;

[0070] Figure 13 A schematic diagram illustrating the selection of a node provided in an embodiment of this application;

[0071] Figure 14A schematic diagram of a node assembly provided in an embodiment of this application;

[0072] Figure 15 A logical diagram of a dragon-stealing scenario provided in this application embodiment;

[0073] Figure 16 A schematic diagram illustrating the script construction for a dragon-stealing scenario, provided as an embodiment of this application;

[0074] Figure 17 This is a schematic diagram of the structure of an action prediction model provided in an embodiment of this application;

[0075] Figure 18 A schematic diagram illustrating the adjustment of model parameters provided in an embodiment of this application;

[0076] Figure 19 A schematic diagram illustrating yet another method of adjusting model parameters provided in an embodiment of this application;

[0077] Figure 20 A step for obtaining an agent network model provided in an embodiment of this application;

[0078] Figure 21 A structural diagram of a control device for an intelligent agent provided in an embodiment of this application;

[0079] Figure 22 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are only some embodiments of the technical solutions of this application, and not all embodiments. Based on the embodiments recorded in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the technical solutions of this application.

[0081] To facilitate a better understanding of the technical solutions of this application by those skilled in the art, some of the terms involved in this application are introduced below.

[0082] Intelligent agents, also known as non-playerable characters (NPCs) or AI characters, are virtual characters or entities controlled by the game engine in a game, rather than being directly manipulated by the player. Intelligent agents can simulate behavior and reactions through programming and algorithms, playing various roles in the game, possessing a degree of autonomy, and capable of reacting to the environment or the player's actions.

[0083] Scenario missions: These are tasks that need to be completed in the target game to determine the outcome of the match. For example, in a game, there are usually ways to determine the winner, and scenario missions refer to the conditions that need to be met to win. Scenario missions can vary in different games and game scenarios. For instance, in some scenarios, a scenario mission might be to destroy the enemy's crystal, while in others, a mission might be to simulate planting and successfully detonating a demolition tool. These are just a few examples.

[0084] Stage tasks: These refer to the tasks that need to be completed in each game stage when fulfilling the script's objectives. Stage tasks may be the same or different across different game stages. Taking the objective of destroying the enemy's crystal as an example, in the first game stage, the corresponding stage task is to control the agent to go to the corresponding lane and engage the enemy hero. In other game stages, the corresponding stage tasks might be roaming, providing support, or securing objectives like dragons.

[0085] The embodiments of this application relate to artificial intelligence and machine learning (ML) technologies, and are primarily designed based on machine learning in artificial intelligence.

[0086] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0087] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large-scale models or foundational models, can be widely applied to downstream tasks across various AI fields after fine-tuning. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0088] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and pre-trained learning. Pre-trained models are the latest development in deep learning, integrating all of these techniques.

[0089] Artificial Neural Networks (ANNs) abstract the neural network of the human brain from an information processing perspective, establishing a simple model and forming different networks with different connection methods. A neural network is a computational model composed of a large number of interconnected nodes (or neurons). Each node represents a specific output function called the activation function. The connection between any two nodes represents a weighted value for the signal passing through that connection, called a weight. This is equivalent to the memory of the artificial neural network. The network's output varies depending on the connection method, weight values, and activation functions. The network itself is usually an approximation of a certain algorithm or function in nature, or it may be an expression of a logical strategy.

[0090] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, digital twins, virtual humans, robots, AI-generated content (AIGC), conversational interaction, smart healthcare, smart customer service, and game AI. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0091] The solution provided in this application involves using machine learning to train an action prediction model for predicting the action set of an agent. In a real game scenario, for each game stage, this action prediction model is used to predict the agent's actions. Further, after predicting the agent's action set, the predicted action set is judged according to pre-determined agent behavior control logic to determine whether the agent can execute the action set in the current scenario. If the agent meets the execution conditions corresponding to the action set, it executes the action; otherwise, at least one action in the action set is adjusted according to the intervention command corresponding to the execution conditions in the behavior control logic to complete the stage character of that game stage and ultimately achieve the target game's script task.

[0092] The word “exemplary” as used below means “serving as an example, embodiment, or illustration.” Any embodiment illustrated as an “exemplary” need not be construed as superior to or better than other embodiments.

[0093] The terms "first" and "second" used in this document are for descriptive purposes only and should not be construed as indicating relative importance or implying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0094] The design concept of the embodiments of this application is briefly introduced below:

[0095] In existing technologies, in order to improve the gaming experience of real players, the design schemes for game intelligent agents typically include behavior tree schemes and reinforcement learning schemes.

[0096] The so-called behavior tree approach refers to decomposing the behavioral decision-making process of an agent into multiple simple tasks, each of which can be achieved through different types of nodes in the behavior tree. However, the combination of nodes in the behavior tree approach is pre-defined, and the agent can only execute behavioral decisions corresponding to the combinations of nodes in the behavior tree. This can lead to behavioral rigidity, resulting in the agent behaving inhumanly in practical business applications. Furthermore, as the agent's behavioral decisions become increasingly complex, more nodes need to be set in the behavior tree, leading to an explosive growth in the number of nodes, resulting in low development efficiency and difficulty in maintenance.

[0097] Reinforcement learning, in essence, uses an intelligent agent model to simulate a real player. A reward mechanism is designed based on the game's script tasks to allocate resources to the agent and thus facilitate the completion of those tasks. However, if this reward mechanism is flawed, it can lead to excessive resource allocation to the real player, resulting in a decreased human-likeness and strength of the agent, ultimately negatively impacting the player's experience.

[0098] Based on this, embodiments of this application provide a control method, apparatus, device, and storage medium for an intelligent agent. In each game stage, based on the current scene description information, character attribute information, and the stage task of the game stage, a set of actions to be executed by the intelligent agent in that game stage is determined. Further, the game state information of the intelligent agent is determined. When the game state information of the intelligent agent meets the execution conditions of the action set, the intelligent agent is controlled to execute the action set; otherwise, at least one action in the action set needs to be adjusted according to an intervention instruction. Based on the above control method, the intelligent agent can ensure its acquisition of game resources, thereby improving its strength, making the intelligent agent's performance closer to that of a real player, and enhancing the gaming experience of real players.

[0099] Furthermore, in this application, the script task is constructed by selecting and assembling decision nodes and intervention nodes on the script building page. The decision and intervention nodes in the script building interface are configurable and draggable, improving node reusability and script building efficiency.

[0100] Furthermore, this application predicts the actions of an agent based on an action prediction model, which includes multiple output modules that predict action sets for different task types. During training, if a new task type is added, only the corresponding output module needs to be added to the output layer of the action prediction model, and the parameters of the corresponding output module need to be tuned. There is no need to adjust other network parameters in the action prediction model, which can significantly reduce the training cost of the model.

[0101] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0102] The solution provided in this application can be applied to most virtual scene applications, such as game application scenarios involving multiple virtual characters, including intelligent agents. Figure 1The diagram illustrates an application scenario in this embodiment. This scenario includes a terminal device 110 and a server 120. Each terminal device 110 can have a client installed corresponding to the server 120, such as a game application supporting a 3D virtual scene. This game application can be any of the following: Virtual Reality (VR) game, Third-Person Shooter (TPS), First-Person Shooter (FPS), Multiplayer Online Battle Arena (MOBA), Massive Multiplayer Online Role-Playing Game (MMORPG), or Multiplayer Gunfight Survival Game. The terminal device 110 and the server 120 can communicate via a communication network. In an optional embodiment, the communication network can be a wired network or a wireless network; therefore, the terminal device 110 and the server 120 can be directly or indirectly connected via wired or wireless communication. This embodiment does not impose specific limitations on this connection.

[0103] In this embodiment, the terminal device 110 is an electronic device used by the target. This electronic device can be a personal computer, mobile phone, tablet computer, laptop, e-book reader, intelligent voice interaction device, smart home device, vehicle terminal, or other computer device with certain computing power that runs instant messaging software and websites or social networking software and websites. The server 120 is used to provide backend services for the client. The server 120 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, cloud functions, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0104] It should be noted that the control method of the intelligent agent in each embodiment of this application can be executed by an electronic device, which can be a terminal device 110 or a server 120. That is, the method can be executed by the terminal device 110 or the server 120 alone, or by the terminal device 110 and the server 120 together.

[0105] Both server 120 and terminal device 110 may include one or more processors, memory, and I / O interfaces for interaction. Furthermore, server 120 may be configured with a database to store data related to the target game, such as model data, scene information, and character information. The memory of server 120 and terminal device 110 may also store program instructions required for execution in the control method of the intelligent agent provided in this application embodiment. These program instructions, when executed by the processor, can be used to implement the control process of the intelligent agent provided in this application embodiment.

[0106] When playing the target game, different players can run the game client through their respective terminal devices 110. The applications running on different terminal devices 110 can be the same, or the same type of application on different operating system platforms. When the terminal device 110 runs the game client, a game interface containing virtual characters is displayed. Furthermore, in this embodiment, the intelligent agent does not have a real player, and therefore does not have a corresponding game interface. However, from the perspective of the intelligent agent, it can still include a corresponding imaginary game screen. Taking a player character and an intelligent agent as an example, when the player character and the intelligent agent are in the same game, the terminal device displays a game interface containing the player character. The player character may be near an intelligent agent, and the two may be in different teams, different organizations, or two hostile groups. The intelligent agent can be controlled through the behavioral control logic of each game stage in the scripted task.

[0107] It should be noted that, Figure 1 The examples shown are merely illustrative; in reality, the number of terminal devices and servers is unlimited and is not specifically limited in the embodiments of this application.

[0108] The following describes the control method of the intelligent agent provided by the exemplary embodiments of this application in conjunction with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way in this respect.

[0109] See Figure 2 The diagram shown is a flowchart of an implementation method for controlling an intelligent agent according to an embodiment of this application. The specific implementation process of this method is as follows:

[0110] Step S201: In response to the agent addition operation triggered for the target game, obtain the script task corresponding to each of the at least one agent to be added.

[0111] In this embodiment, the target game can be any of the following: a shooting game, a tactical competitive game, a role-playing game, or a survival game. In practical applications, when the target game is launched, at least one intelligent agent can be loaded into the virtual scene of the target game. Specifically, when a real player selects the human-computer battle mode, or launches the game test mode, the target game is launched, and at least one intelligent agent is loaded. In some scenarios, the virtual game scene can be any virtual scene, such as a 1V1 game scene, a 2V2 game scene, or a 5V5 game scene. In some scenarios, different intelligent agents may correspond to the same or different script tasks.

[0112] For example, taking a real player starting a player-versus-AI match and selecting a 2v2 game scenario, such as... Figure 3 As shown, in response to the start of the target game, the game character of the real player, along with three intelligent agents, are loaded into the virtual game scene. The scenario mission for agent 1 is to defeat crystal 1, while the scenario missions for agents 2 and 3 are to defeat crystal 2.

[0113] Step S202, for each agent in each game phase of the target game, perform the following operations respectively:

[0114] Step S2021: Based on the scene description information of the stage scene corresponding to a game stage, the role attribute information of an agent in a game stage, and the stage task corresponding to the script task of an agent in the game stage, determine the set of actions to be executed by an agent in a game stage.

[0115] In some embodiments, the task type corresponding to a stage task can be obtained from multiple task types based on the scene description information of the stage scene corresponding to a game stage. Next, scene features can be extracted from the scene description information to obtain corresponding scene features; and character feature extraction can be performed from the character attribute information to obtain corresponding character attribute features. Furthermore, based on the task type corresponding to the stage task, combined with the scene features and character attribute features, the set of actions an agent will perform in a game stage can be predicted. For example, task types may include navigation, scheduling, support, team battles, etc.

[0116] As an example, there is a correlation between scene description information and task types; different task types are associated with different scene description information. In the current game phase, the task type corresponding to the scene description information can be determined from multiple task types using the scene description information of the current game phase's scene. For example, in a 5v5 game scenario, when an agent is in the first game phase after the game starts, the scene description information can determine the agent's task type as roaming to the corresponding lane. Alternatively, when a team fight breaks out in a certain game phase, the scene description information can determine the corresponding task type as providing support.

[0117] like Figure 4 As shown, in a game phase, after determining the current scene description information, it can be matched with each of the preset scene description information to determine the target scene description information. For example, if there are 10 preset scene description information, the current scene description information is matched with each of the 10 preset scene description information. Specifically, the similarity between the current scene description information and the 10 preset scene description information can be calculated, and the scene description information with the highest similarity among the 10 preset scene description information is taken as the target scene description information. For example, if the preset scene description information 3 has the highest similarity, then preset scene description information 3 is taken as the target scene description information. Furthermore, based on the association between each scene description information and each task type, the target task type associated with the target scene description information can be determined as the task type for this game phase. For example, if the task type associated with preset scene description information 3 is task type 3, then task type 3 is taken as the task type corresponding to the phase task in the current game phase.

[0118] In some embodiments, taking the mid-lane of a 5V5 competitive game as an example, in a game phase, the game scenario associated with the laning task type of the mid-lane game character is as follows: Figure 5 As shown, each character (including the AI ​​agent and the real player's game character) is distributed on the corresponding lane, and the enemy's characters on each lane have vision of each other.

[0119] In some embodiments, the game scenario associated with the support mission type is: the number of enemy characters within a set range of any given game character is greater than the number of teammates. In other scenarios, the game scenario associated with the support mission type may also be: an enemy mid-lane character disappears from mid-lane vision, or it is determined that the enemy mid-lane character's vision has shifted to the bottom lane or top lane, etc. See also Figure 5 As shown, within the area indicated by the dotted line in the bottom lane, there are two teammates and three enemy characters. If the number of enemies is greater than the number of teammates, then it is determined that teammates need to provide support. In this case, the game scenario corresponds to the support mission type.

[0120] In some embodiments, the game scenario associated with the "Dragon-Slaying" mission type is that a large dragon exists within a designated area, and teammates attack the dragon within a designated map area. See also Figure 5 As shown, if a large dragon exists within the map area corresponding to the dotted line, and teammates are attacking the large dragon within the map area corresponding to the dotted line, then the game scene is determined to be the game scene corresponding to the dragon-slaying mission type.

[0121] Furthermore, if in this game phase, the game scenario corresponding to agent 11 is as follows: Figure 6 As shown, the real player's game character and other intelligent agents are all in their respective lanes. Furthermore, the game scene corresponding to agent 11 can be... Figure 5 The similarity of the game scenes associated with each task type is calculated to obtain the similarity score of the game scenes corresponding to each task type. For example, if the similarity score with the game scene of the laning task type is 90%, the similarity score with the game scene of the support task type is 50%, and the similarity score with the game scene of the dragon-slaying task type is 30%, then the game scene of the laning task type is determined to have the highest similarity score, and the laning task type is determined to be the task type corresponding to the current game stage.

[0122] Step S2022: Obtain game state information of an agent. The game state information includes: the current role state of an agent, and the current acquisition status of an agent for reward resources.

[0123] In some embodiments, the current role state of the intelligent agent includes health, skill attributes, and movement direction. Reward resources refer to pre-set game resources in the game scene. By acquiring reward resources, the intelligent agent's attributes can be directly or indirectly enhanced. For example, if the reward resource is a minion wave, the intelligent agent can gain gold by killing minions or jungle monsters and purchase equipment to improve its attributes and level. Furthermore, reward resources can also be other resources that refresh according to set rules; this application does not specifically limit this.

[0124] Specifically, the current character state of the AI ​​can be displayed in the game interface. For example... Figure 7 As shown, in the game interface, the health bar can be displayed above the game character and moves with the character. The first designated area of ​​the game interface can be used to control the movement direction of the AI; the direction the AI ​​is facing is its movement direction. For example... Figure 7In the bottom left corner, four triangles are used to control the agent's movement in different directions. Additionally, the second settings area of ​​the game interface displays components related to the agent's skills. By clicking on the corresponding component, you can control the agent to execute the skill action associated with that component, inflicting corresponding damage on game characters or reward resources within a set range.

[0125] Furthermore, clicking on the corresponding skill component will display the cooldown time for the next use of that skill within that component. For example... Figure 7 In the bottom right corner, the component with the number 1 indicates that the skill has 1 second of cooldown remaining, 3 indicates that the skill has 3 seconds of cooldown remaining, and 5 indicates that the skill has 5 seconds of cooldown remaining. When the number is not displayed, it means that the skill has been restored and can be used directly without waiting for cooldown.

[0126] In some scenarios, the agent's current acquisition status of game resources can be reflected by the agent's game level. The game level can be displayed above the game character along with the health bar and move with the agent, such as... Figure 8 As shown, the "10" above the agent indicates that the agent's game level is 10. Furthermore, the agent's acquisition of reward resources can be determined by its economic growth during this stage. For example, if the reward resource is a minion wave, it's necessary to determine whether the minion wave has spawned and whether the agent has killed it during this game stage. If the agent has acquired the minion wave, the agent will experience corresponding growth during this game stage, and thus, the acquisition of reward resources can be determined by monitoring the agent's economic growth.

[0127] Step S2023: When the current character state and the current acquisition situation both meet the execution conditions of the action set, control an agent to execute the action set in a game phase.

[0128] In some embodiments, the behavioral control logic of an agent in a game phase can be obtained, and the execution conditions in the behavioral control logic can be determined. The execution conditions include at least one state execution sub-condition and at least one resource execution sub-condition. Each state execution sub-condition represents a state requirement that an agent's role state must satisfy, and each resource execution sub-condition represents a resource acquisition requirement that an agent's resource acquisition situation must satisfy.

[0129] For example, the status execution sub-condition could be that health exceeds a set threshold, skill cooldown recovery, etc., while the resource execution sub-condition could be that there are no remaining minions on the lane, or that all jungle monsters in a set area on the map have been killed, etc. This application does not specify these limitations. Specifically, the status execution sub-condition could be that health exceeds 30%, or that skill cooldown recovery time is less than a set duration, such as skill cooldown recovery time being less than 5 seconds. The resource execution sub-condition could be that there are no minions on the lane, or that all jungle monsters in the jungle area corresponding to the agent have been killed, and that the time until the next respawn is less than 30 seconds.

[0130] Furthermore, if the number of state execution sub-conditions satisfied by the current role state reaches a first threshold, and the number of resource execution sub-conditions satisfied by the current acquisition situation reaches a second threshold, it is determined that both the current role state and the current acquisition situation satisfy the execution conditions for the action set. The first and second thresholds may be the same or different; this application does not impose specific limitations on this.

[0131] As an example, if the execution conditions include four state execution sub-conditions and two resource execution sub-conditions, when the agent's current role state satisfies three of the state execution sub-conditions, the current acquisition situation satisfies two of the resource execution sub-conditions, and both the first and second quantity thresholds are 2, then it is determined that both the current role state and the current acquisition situation satisfy the execution conditions of the action set, and the agent is controlled to execute the action set.

[0132] Based on the above scheme, after determining the set of actions to be executed by the agent, it will be determined whether the agent meets the execution conditions based on the current character status and the current acquisition of reward resources. Then, when the execution conditions are met, the corresponding set of actions will be executed. This can ensure the agent's acquisition of game resources, thereby improving its own strength, making the agent's performance closer to that of real players, and enhancing the gaming experience of real players.

[0133] In some embodiments, step S2024 may be included after step S2023:

[0134] When the game state information of an agent does not meet the execution conditions of the action set, the agent adjusts at least one action in the action set based on the intervention instructions corresponding to the execution conditions.

[0135] Specifically, the game state information of an agent does not meet the execution conditions for executing a set of actions, which can be divided into the following three cases:

[0136] Case 1: When the number of state execution sub-conditions satisfied by the current role state of the agent is less than the first quantity threshold, it is determined that the game state information of the agent does not meet the execution conditions of the action set.

[0137] As an example, when the current role state of the agent satisfies two state execution sub-conditions and the first quantity threshold is 3, regardless of whether the number of resource execution sub-conditions satisfied by the agent's current acquisition situation is greater than the second quantity threshold, it is considered that the agent's game state information does not meet the execution conditions of the action set.

[0138] Case 2: When the number of resource execution sub-conditions satisfied by the agent's current acquisition is less than the second quantity threshold, it is determined that the agent's game state information does not meet the execution conditions of the action set.

[0139] For example, if the agent's current acquisition situation satisfies one resource execution sub-condition and the second quantity threshold is 2, then regardless of whether the number of state execution sub-conditions satisfied by the agent's current role state is greater than the first quantity threshold, it is determined that the agent's game state information does not meet the execution conditions of the action set.

[0140] Case 3: When the number of state execution sub-conditions satisfied by the agent's current role state is less than the first quantity threshold, and the number of resource execution sub-conditions satisfied by the agent's current acquisition state is less than the second quantity threshold, it is determined that the agent's game state information does not meet the execution conditions of the action set.

[0141] For example, if the agent's current role state satisfies two state execution sub-conditions, the current acquisition situation satisfies one resource execution sub-condition, the first quantity threshold is 3, and the second quantity threshold is 2, then the agent's current role state and current acquisition situation do not satisfy the corresponding quantity thresholds, and it is determined that the agent's game state information does not meet the execution conditions for executing the action set.

[0142] In some embodiments, each execution condition corresponds to an intervention instruction. Different execution conditions may correspond to different intervention instructions or the same intervention instruction; this application does not specifically limit this. When it is determined that the game state information of the agent does not meet the execution conditions, at least one execution action contained in the execution action set can be adjusted based on the intervention instruction corresponding to the unmet execution condition.

[0143] As an example, suppose there are 5 execution conditions, denoted as Execution Condition 1, Execution Condition 2, Execution Condition 3, Execution Condition 4, and Execution Condition 5. Each execution condition corresponds to a different intervention instruction, such as... Figure 9 As shown. If the game state information of the agent fails to meet the execution conditions due to the failure to meet execution conditions 1 and 3, then the intervention instructions 1 and 3 corresponding to execution conditions 1 and 3 are determined, and then at least one execution action contained in the execution action set is adjusted through intervention instructions 1 and 3.

[0144] For example, suppose there are 5 execution conditions, denoted as execution condition 1, execution condition 2, execution condition 3, execution condition 4, and execution condition 5. Different execution conditions may correspond to the same intervention instruction, such as... Figure 10 As shown. If the game state information of the agent fails to meet the execution conditions due to the failure to meet execution conditions 1 and 3, then the intervention instruction 11 corresponding to execution conditions 1 and 3 is determined, and then at least one execution action contained in the execution action set is adjusted through the intervention instruction 11.

[0145] In some embodiments, different execution conditions can be judged sequentially in a preset order. If both the first and second quantity thresholds are 1, then when it is determined that the game state information does not meet an execution condition, at least one execution action contained in the execution action set is adjusted based on the intervention instruction corresponding to the unmet execution condition.

[0146] For example, suppose there are 5 execution conditions, and the preset order is execution condition 1, execution condition 2, execution condition 3, execution condition 4, and execution condition 5. Figure 11 As shown, the game state information can be judged in a preset order to determine whether it meets the five execution conditions. If the game state information meets execution condition 1 and execution condition 2, but does not meet execution condition 3, the intervention instruction 33 corresponding to execution condition 3 can be used to adjust at least one execution action in the execution action set.

[0147] Based on the above scheme, when it is determined that the game state information of the agent does not meet the execution conditions of the action execution set, the execution actions in the action execution set can be intervened according to the intervention instructions corresponding to the execution conditions. This allows the agent to execute the corresponding action set only when its own state allows it and it can obtain resources, thereby ensuring the strength of the agent and providing a better gaming experience for real players.

[0148] In some embodiments, the script task corresponding to each agent is obtained in the following way:

[0149] Obtain the preset game stages; for a given agent, determine the behavioral control logic of the agent in each game stage; based on the preset execution order between game stages, combine the behavioral control logic corresponding to each game stage to obtain the script task of an agent.

[0150] As an example, if the target game includes three game stages: A, B, and C, then for a given agent, the behavioral control logic for that agent in each of the three game stages is determined. Furthermore, after determining the behavioral control logic for each of the three game stages, if the execution order of the three game stages A, B, and C is ABC, then the behavioral control logic for stage A → stage B → stage C is combined to obtain the agent's script task.

[0151] In some embodiments, the behavioral control logic of an agent in a game phase is determined in the following way:

[0152] The script building interface displays a pool of decision nodes and a pool of intervention nodes. Each decision node represents an execution condition, and each execution condition includes the parameter requirements that trigger the corresponding agent to execute the set of corresponding actions. Each intervention node represents an intervention instruction, and each intervention instruction is used to instruct adjustments to the set of actions corresponding to the agent. Figure 12 The diagram shows the script building interface, which includes a judgment node pool and an intervention node pool. The judgment node pool includes health judgment nodes, enemy lock judgment nodes, dragon attack judgment nodes, target range nodes, etc., while the intervention node pool can include skill intervention, movement intervention, delay intervention, etc., specifically including target offset nodes, skill offset nodes, skill disabling nodes, target selection disabling nodes, etc. This application integrates the Unity editor into game intelligent agent development, building a visual UI editing interface. Developers can construct reusable judgment and intervention nodes, and then quickly build the behavior control logic for each stage of the task based on the visual UI interface. The development method is simple and clear, thereby improving development efficiency.

[0153] Furthermore, in response to the selection operation for the judgment node pool and the intervention node pool, at least one selected target judgment node and at least one target intervention node are presented in the script building interface.

[0154] Continuing with the example above, in response to operations such as selecting a health assessment node and a target range node from the assessment node pool, and selecting a target forbidden node from the intervention node pool, the corresponding nodes are displayed on the scenario building interface, such as... Figure 13 As shown.

[0155] In response to the assembly operation, at least one target judgment node and at least one target intervention node are assembled according to the indicated assembly sequence to obtain the behavioral control logic of an agent in a game phase.

[0156] Specifically, in response to a drag-and-drop operation on at least one target decision node, the assembly order among at least one target decision node can be determined. For example, multiple target decision nodes can be dragged and dropped to determine their assembly order. In response to a connection operation between any two adjacent target decision nodes, at least one target decision node can be assembled. Specifically, the assembly of multiple target decision nodes can be determined by pointing to the order using arrows.

[0157] For example, three target judgment nodes can be selected from the judgment node pool: health judgment, skill judgment, and player judgment. The health judgment is used to determine whether the agent's health is higher than a set threshold, the skill judgment is used to determine whether the agent's skill cooldown time is less than a set duration, and the player judgment is used to determine whether there are game players within the set range corresponding to the agent.

[0158] Furthermore, after selecting the aforementioned three target judgment nodes, these three target judgment nodes are displayed on the scenario building interface. Furthermore, dragging the display component of any one of the three target judgment nodes responds to dragging operations, thereby changing the display position of the three target judgment nodes in the scenario building interface. Furthermore, an arrow can be used to connect the player judgment node and the health judgment node, such as... Figure 14 As shown, the arrow points to the health assessment node, indicating that the system first checks if a player exists within a set range. If a player is confirmed to exist within the set range, a health assessment is then performed to determine if the agent's current health is higher than a set health threshold. Similarly, an arrow can connect the health assessment node and the skill assessment node, with the arrow pointing to the skill assessment node, indicating that a skill assessment is performed after the health assessment, thus completing the assembly of the three target assessment nodes.

[0159] For each target judgment node, determine the corresponding target intervention node; in response to the connection operation between each target judgment node and its corresponding target intervention node, assemble each target judgment node and its corresponding target intervention node.

[0160] As an example, let's take the player's dragon-stealing mission as a stage task in a game. During the game, the script logic is as follows: Figure 15 As shown, the specific process is as follows:

[0161] Step S1501: Determine whether an epic monster is being attacked. If so, proceed to step S1502; otherwise, end the determination.

[0162] Step S1502: Determine whether the epic monster's health is below a set threshold. If so, proceed to step S1503; otherwise, proceed to step S1501.

[0163] Step S1503: Determine whether there are players within the set range. If so, proceed to step S1504; otherwise, proceed to step S1501.

[0164] Step S1504: The Dragon Stealing Script is triggered and reported.

[0165] Step S1505: Disable Smite.

[0166] Step S1506: Determine if the epic monster is dead. If so, proceed to step S1507; otherwise, proceed to step S1501.

[0167] Step S1507: Determine if the player has the corresponding buff. If so, proceed to step S1508; otherwise, end the process.

[0168] Step S1508: The Dragon-Snatching Scenario is reported.

[0169] Based on the above logic, the control logic for the intelligent agent in the player's dragon-stealing behavior is as follows: Figure 16 As shown, the corresponding judgment node and intervention node are selected, and then the connection operation between the judgment node and the intervention node is performed to obtain the behavior control logic of the agent when the player steals the dragon.

[0170] In some embodiments, scene features and character attribute features are extracted using a target action prediction model; optionally, in embodiments of this application, see [reference needed]. Figure 17 As shown, the target action prediction model includes a feature extraction module 10 and multiple output modules 11. Different output modules 11 are used to output predicted action sets for different task types.

[0171] Specifically, the aforementioned target action prediction model can be obtained in the following way:

[0172] First, the feature extraction module extracts features from each first sample data of the first task type, and the first output module associated with the first task type outputs the first predicted action set corresponding to each sample feature and the loss value generated based on the first predicted action set. The parameter settings of the feature extraction module and the first output module are then adjusted. The first task type can be any one of multiple task types.

[0173] For example, taking the first task type as the supporting task type, the sample data for the supporting task type includes sample scene description information and sample role attribute information. The feature extraction module 10 can extract features from the sample scene description information and the sample role attribute information respectively to obtain sample scene features and sample role features.

[0174] Furthermore, the output module 1 associated with the support task type can be used to predict actions based on the sample scene features and sample character features to obtain the predicted action set of the sample data for the support task type. Then, the loss value can be determined based on the predicted action set of the sample data for the support task type and the execution action set of the real player corresponding to the sample data for the support task type. Subsequently, the feature extraction module 10 and the output module associated with the support task type can be tuned using the loss value corresponding to the sample data for each support task type, such as... Figure 18 As shown.

[0175] Specifically, a loss value can be determined for the sample data of each supported task type, and the feature extraction module 10 and the output module 1 associated with the supported task type can be tuned based on the loss value for the sample data of each supported task type. In some embodiments, the loss values ​​for the sample data of each supported task type can be weighted to obtain a total loss value, and the feature extraction module 10 and the output module 1 associated with the supported task type can be tuned based on the total loss value.

[0176] Next, for each task type other than the first task type, the following steps are performed: the feature extraction module extracts features from each second sample data of a task type through the parameter-tuned feature extraction module, and outputs the second predicted action set corresponding to each sample feature through other output modules associated with a task type. Based on the loss value generated by the second predicted action set, the parameters of other output modules are tuned.

[0177] For example, such as Figure 19 As shown, taking the team battle mission type as an example, the sample data for the team battle mission type includes sample scene description information and sample character attribute information. First, the adjusted feature extraction module 10 can extract features from each sample data of the team battle mission type, and the output module 2 associated with the team battle mission type can output the predicted action set 2 corresponding to each sample feature of each sample data of the team battle mission type.

[0178] Specifically, for each type of team battle mission, the parameter-tuned feature extraction module 10 can extract features from the sample scene description information and the sample role attribute information to obtain sample scene features and sample role features.

[0179] Furthermore, the output module 12 associated with the team battle mission type can be used to predict actions based on the sample scene features and sample character features, resulting in a predicted action set 2 for the sample data of the team battle mission type. Then, the loss value can be determined based on the predicted action set 2 of the sample data of the team battle mission type and the execution action set of the real players corresponding to the sample data of the team battle mission type. Finally, the output module 2 associated with the team battle mission type can be tuned using the loss value corresponding to the sample data of each team battle mission type.

[0180] In this application, sample datasets for each task type can be constructed through the task feature extraction module X1, and task training can be extended based on the game agent model through the task training module X2. The evaluation module X3 evaluates whether the task metrics of the agent for the corresponding task type meet the target values. Figure 20 As shown.

[0181] Specifically, the task feature extraction module X1 first designs sampling logic to obtain sample datasets for the corresponding task types from real player game data. The game state (including complex game states such as map terrain, multiple target units, and incomplete information) of any task type's sample dataset is then visualized and vectorized. Specifically, the player's surrounding terrain environment (scene description information) is visualized and vectorized, such as surrounding obstacles, hero skill influence range, surrounding bushes, and the distribution of teammates and enemies on the map. Furthermore, player's own attribute information, enemy and teammate attributes, and shared attribute features are vectorized, such as health, hero buffs, skill attributes, and movement direction.

[0182] Furthermore, the task training module X2 extends task training based on the game agent model. By designing independent output modules for different task types, it enables incremental training for new task types, achieving parallel training of multiple task types without affecting the capabilities of the original model. During training, the original model's network parameters remain unchanged; only the model parameters of the output modules corresponding to the newly added task types are updated.

[0183] Next, the game agent trained by the task training module X2 needs to be connected to the evaluation module X3 to evaluate whether the task metrics for the corresponding task type meet the target values. Specifically, the evaluation module X3 constructs a self-play process, randomly selecting 10 heroes to form a game lineup. The 10 heroes then play against each other to generate game data, which is then analyzed to evaluate the agent's human-likeness and task performance capabilities. Specifically, 3000 self-play games are conducted to ensure data reliability.

[0184] With the above approach, during the training of the action prediction model, only the network parameters of the feature extraction module need to be adjusted during the first training. If a new task type is added, only the corresponding output module needs to be added to the output layer of the action prediction model and the parameters of the corresponding output module need to be tuned. There is no need to adjust other network parameters in the action prediction model, which allows multiple training tasks to be executed in parallel and can significantly reduce the training cost of the model.

[0185] Based on the same technical concept, embodiments of this application also provide a control device for an intelligent agent. For example... Figure 21 The diagram shown is a structural schematic of the control device 2100 for the intelligent agent. This device 2100 can execute any step of the aforementioned control method for the intelligent agent; to avoid repetition, it will not be described in detail here. The device 2100 may include:

[0186] The acquisition unit 2101 is used to acquire the script task corresponding to each of at least one agent to be added in response to an agent addition operation triggered for the target game.

[0187] Processing unit 2102 is configured to perform the following operations for each agent in each game phase of the target game:

[0188] Based on the scene description information of the stage scene corresponding to a game stage, the role attribute information of an agent in the game stage, and the stage task corresponding to the script task of the agent in the game stage, the set of actions to be executed by the agent in the game stage is determined.

[0189] The game state information of the intelligent agent is obtained by the acquisition unit 2101. The game state information includes: the current character state of the intelligent agent, and the current acquisition status of the intelligent agent for reward resources.

[0190] When both the current character state and the current acquisition status meet the execution conditions of the action set, the agent is controlled to execute the action set in the game phase.

[0191] In one possible implementation, the processing unit 2102 is specifically used for:

[0192] Obtain the preset game stages;

[0193] For a given intelligent agent, determine the behavioral control logic of the intelligent agent in each game phase;

[0194] Based on the preset execution order between the game stages, the behavioral control logic corresponding to each game stage is combined to obtain the script task of the intelligent agent.

[0195] In one possible implementation, the processing unit 2102 is specifically used for:

[0196] A script building interface is presented, which displays a judgment node pool and an intervention node pool. Each judgment node represents an execution condition, and each execution condition includes: parameter requirements for triggering the corresponding agent to execute the corresponding set of execution actions. Each intervention node represents an intervention instruction, and each intervention instruction is used to instruct the corresponding set of execution actions of the corresponding agent to be adjusted.

[0197] In response to the selection operation for the judgment node pool and the intervention node pool, at least one selected target judgment node and at least one target intervention node are presented in the script building interface;

[0198] In response to the assembly operation, the at least one target judgment node and the at least one target intervention node are assembled according to the indicated assembly sequence to obtain the behavioral control logic of the agent in a game phase.

[0199] In one possible implementation, the processing unit 2102, when assembling the at least one target judgment node and the at least one target intervention node according to the indicated assembly order in response to the assembly operation, is specifically configured to:

[0200] In response to a drag operation on the at least one target judgment node, the assembly order among the at least one target judgment node is determined;

[0201] In response to a connection operation between any two adjacent target decision nodes, assembly is performed between the at least one target decision node;

[0202] For each of the target judgment nodes, determine the corresponding target intervention node;

[0203] In response to the connection operation between each target judgment node and its corresponding target intervention node, each target judgment node and its corresponding target intervention node are assembled.

[0204] In one possible implementation, the processing unit 2102 is further configured to:

[0205] When the game state information of an agent does not meet the execution conditions of the set of actions, at least one action in the set of actions is adjusted based on the intervention instruction corresponding to the execution conditions.

[0206] In one possible implementation, when the processing unit 2102 determines the set of actions to be performed by the intelligent agent in a game stage based on the scene description information of the stage scene corresponding to a game stage, the role attribute information of an intelligent agent in the game stage, and the stage task corresponding to the script task of the intelligent agent in the game stage, it is specifically used to:

[0207] Based on the scene description information of the stage scene corresponding to the game stage, the task type corresponding to the stage task is obtained from multiple task types;

[0208] Scene features are extracted from the scene description information to obtain the corresponding scene features;

[0209] The character attribute information is subjected to character feature extraction to obtain the corresponding character attribute features;

[0210] Based on the task type corresponding to the stage task, combined with the scene features and the character attribute features, the set of actions to be performed by the agent in the game stage is predicted.

[0211] In one possible implementation, the scene features and the character attribute features are extracted using a target action prediction model; the target action prediction model includes a feature extraction module and multiple output modules, with different output modules used to output predicted action sets for different task types; the processing unit 2102 is further configured to obtain the target action prediction model in the following manner:

[0212] The feature extraction module extracts features from each first sample data of the first task type, and the first output module associated with the first task type outputs the first predicted action set corresponding to each sample feature and the loss value generated based on the first predicted action set. The feature extraction module and the first output module are then tuned. The first task type can be any one of multiple task types.

[0213] For each task type other than the first task type, the following steps are performed: feature extraction is performed on each second sample data of a task type through the parameter-tuned feature extraction module; the second predicted action set corresponding to each sample feature is output through other output modules associated with the task type; and the other output modules are parameter-tuned based on the loss value generated by the second predicted action set.

[0214] In one possible implementation, when the processing unit 2102 determines that both the current role state and the current acquisition situation satisfy the execution conditions of the action set, it is specifically used for:

[0215] The behavior control logic of the agent in a game phase is obtained, and the execution conditions in the behavior control logic are determined; wherein, the execution conditions include at least one state execution sub-condition and at least one resource execution sub-condition, each state execution sub-condition representing: the state requirement of a set state that the role state of the agent must meet, and each resource execution sub-condition representing: the resource acquisition situation of the agent must meet a set resource acquisition requirement;

[0216] If the number of state execution sub-conditions satisfied by the current role state reaches a first threshold, and the number of resource execution sub-conditions satisfied by the current acquisition situation reaches a second threshold, then it is determined that both the current role state and the current acquisition situation satisfy the execution conditions of the execution action set. For ease of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware.

[0217] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0218] Having introduced the control method and apparatus for an intelligent agent according to exemplary embodiments of this application, we will now introduce an electronic device according to another exemplary embodiment of this application.

[0219] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0220] Based on the same technical concept as the above-described method embodiments, this application also provides an electronic device. In one embodiment, the electronic device may be a server, such as... Figure 1 The server 120 is shown. In this embodiment, the structure of the electronic device can be as follows: Figure 22 As shown, it includes a memory 2201, a communication module 2203, and one or more processors 2202.

[0221] The memory 2201 is used to store computer programs executed by the processor 2202. The memory 2201 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0222] Memory 2201 may be volatile memory, such as random-access memory (RAM); memory 2201 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 2201 may be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 2201 may be a combination of the above-described memories.

[0223] Processor 2202 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 2202 is used to implement the control method of the intelligent agent described above when calling the computer program stored in memory 2201.

[0224] The communication module 2203 is used to communicate with terminal devices and other servers.

[0225] This application embodiment does not limit the specific connection medium between the memory 2201, communication module 2203, and processor 2202. This application embodiment... Figure 22 The memory 2201 and the processor 2202 are connected via a bus 2204, and the bus 2204 is in Figure 22 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 2204 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 22 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0226] The memory 2201 stores a computer storage medium, which stores computer-executable instructions for implementing the control method of the intelligent agent according to the embodiments of this application. The processor 2202 is used to execute the aforementioned control method of the intelligent agent, such as... Figure 2 As shown.

[0227] In some possible implementations, various aspects of the intelligent agent control method provided in this application can also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program is used to cause the electronic device to perform the steps in the intelligent agent control method according to the various exemplary embodiments of this application described above. For example, the electronic device can perform actions such as... Figure 2 The steps are shown in the figure.

[0228] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0229] The program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include a computer program, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0230] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0231] Computer programs contained on readable media may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0232] Computer programs for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The computer program can execute entirely on the user's electronic device, partially on the user's electronic device, as a standalone software package, partially on the user's electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the user's electronic device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0233] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0234] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0235] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0236] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0237] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0238] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0239] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A control method for an intelligent agent, characterized in that, include: In response to an agent addition operation triggered for the target game, obtain the script tasks corresponding to at least one agent to be added; For each agent in each game phase of the target game, perform the following operations: Based on the scene description information of the stage scene corresponding to a game stage, the role attribute information of an agent in the game stage, and the stage task corresponding to the script task of the agent in the game stage, the set of actions to be executed by the agent in the game stage is determined. Obtain the game state information of the intelligent agent, the game state information including: the current character state of the intelligent agent, and the current acquisition status of the intelligent agent for reward resources; When both the current character state and the current acquisition status meet the execution conditions of the action set, the agent is controlled to execute the action set in the game phase.

2. The method as described in claim 1, characterized in that, The script task corresponding to each of the aforementioned agents is obtained in the following manner: Obtain the preset game stages; For a given intelligent agent, determine the behavioral control logic of the intelligent agent in each game phase; Based on the preset execution order between the game stages, the behavioral control logic corresponding to each game stage is combined to obtain the script task of the intelligent agent.

3. The method as described in claim 2, characterized in that, The behavior control logic of an agent in a game phase is determined in the following way: The script building interface is presented, which displays a judgment node pool and an intervention node pool. Each judgment node represents an execution condition, and each execution condition includes: parameter requirements for triggering the corresponding agent to execute the corresponding set of execution actions. Each intervention node represents an intervention instruction, and each intervention instruction is used to instruct the adjustment and processing of the execution action set corresponding to the corresponding agent. In response to the selection operation for the judgment node pool and the intervention node pool, at least one selected target judgment node and at least one target intervention node are presented in the script building interface; In response to the assembly operation, the at least one target judgment node and the at least one target intervention node are assembled according to the indicated assembly order to obtain the behavior control logic of the agent in a game phase.

4. The method as described in claim 3, characterized in that, The assembly operation, in response to the assembly operation, assembling the at least one target judgment node and the at least one target intervention node according to the indicated assembly sequence, includes: In response to a drag operation on the at least one target judgment node, the assembly order among the at least one target judgment node is determined; In response to a connection operation between every two adjacent target decision nodes, assembly is performed between the at least one target decision node; For each of the target judgment nodes, determine the corresponding target intervention node; In response to the connection operation between each target judgment node and its corresponding target intervention node, each target judgment node and its corresponding target intervention node are assembled.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: When the game state information of an agent does not meet the execution conditions of the set of actions, at least one action in the set of actions is adjusted based on the intervention instruction corresponding to the execution conditions.

6. The method according to any one of claims 1-4, characterized in that, Based on the scene description information of a game stage, the role attribute information of an agent in that game stage, and the stage task corresponding to the agent's script task in that game stage, the set of actions to be executed by the agent in that game stage is determined, including: Based on the scene description information of the stage scene corresponding to the game stage, the task type corresponding to the stage task is obtained from multiple task types; Scene features are extracted from the scene description information to obtain the corresponding scene features; The character attribute information is subjected to character feature extraction to obtain the corresponding character attribute features; Based on the task type corresponding to the stage task, combined with the scene features and the character attribute features, the set of actions to be performed by the agent in the game stage is predicted.

7. The method as described in claim 6, characterized in that, The scene features and the character attribute features are extracted using a target action prediction model; the target action prediction model includes a feature extraction module and multiple output modules, with different output modules used to output predicted action sets for different task types; the target action prediction model is obtained in the following way: The feature extraction module extracts features from each first sample data of the first task type, and the first output module associated with the first task type outputs the first predicted action set corresponding to each sample feature and the loss value generated based on the first predicted action set. The feature extraction module and the first output module are then tuned. The first task type can be any one of multiple task types. For each task type other than the first task type, the following steps are performed: feature extraction is performed on each second sample data of a task type through the parameter-tuned feature extraction module; the second predicted action set corresponding to each sample feature is output through other output modules associated with the task type; and the other output modules are parameter-tuned based on the loss value generated by the second predicted action set.

8. The method according to any one of claims 1-4, characterized in that, The determination that both the current role state and the current acquisition status satisfy the execution conditions of the action set includes: The behavior control logic of the agent in a game phase is obtained, and the execution conditions in the behavior control logic are determined; wherein, the execution conditions include at least one state execution sub-condition and at least one resource execution sub-condition, each state execution sub-condition representing: the state requirement of a set state that the role state of the agent must meet, and each resource execution sub-condition representing: the resource acquisition situation of the agent must meet a set resource acquisition requirement; If the number of state execution sub-conditions satisfied by the current role state reaches a first quantity threshold, and the number of resource execution sub-conditions satisfied by the current acquisition situation reaches a second quantity threshold, it is determined that both the current role state and the current acquisition situation satisfy the execution conditions of the execution action set.

9. A control device for an intelligent agent, characterized in that, include: The acquisition unit is used to acquire the script tasks corresponding to at least one agent to be added in response to an agent addition operation triggered for the target game. The processing unit is configured to perform the following operations for each agent in each game phase of the target game: Based on the scene description information of the stage scene corresponding to a game stage, the role attribute information of an agent in the game stage, and the stage task corresponding to the script task of the agent in the game stage, the set of actions to be executed by the agent in the game stage is determined. The game state information of the intelligent agent is obtained by the acquisition unit. The game state information includes: the current character state of the intelligent agent, and the current acquisition status of the intelligent agent for reward resources. When both the current character state and the current acquisition status meet the execution conditions of the action set, the agent is controlled to execute the action set in the game phase.

10. An electronic device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method as described in any one of claims 1 to 8 according to the obtained program instructions.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, The method includes a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of any one of claims 1 to 8.

Images