Game AI-oriented nodal data-driven decision-making method and system

Abstract

The invention discloses a game AI-oriented nodal data-driven decision-making method and system, and relates to the technical field of electronic games. The method comprises the following steps: constructing a game AI neural node library; the graphical nodes are connected and assembled into a graphical decision logic structure; executing the decision structure in game operation to generate an AI instruction, and recording a differentiable track; calculating a performance reward according to the game feedback; and updating the internal network parameters of the nodes in the node library and optimizing the decision logic structure by using the performance reward and the differentiable decision trajectory. The technical problems that end-to-end training cannot be performed and automatic optimization is difficult to perform according to game feedback due to the fact that traditional game AI decision logic depends on a manual compiling rule and is difficult to adapt to a complex dynamic environment are solved, and the purposes that a flexible and adjustable decision logic structure is assembled and constructed through graphical nodes, and the algorithm is simple and convenient to implement are achieved. And end-to-end gradient optimization is realized in combination with differentiable decision tracks and performance reward feedback, and the technical effects of improving the adaptability, intelligence and optimization efficiency of game AI decisions are achieved.

Description

Technical Field

[0001] This invention relates to the field of video game technology, and more specifically to a node-based data-driven decision-making method and system for game AI. Background Technology

[0002] In recent years, with the increasing complexity of video games, higher demands have been placed on the intelligence level of non-player characters (NPCs). Traditional game AI typically relies on state machines, behavior trees, or scripted hard-coded logic. While these methods offer advantages in controllability, they often prove rigid and difficult to adapt to open and dynamic game environments. Developers need to invest significant effort in manually designing and debugging complex rules and state transitions, which is not only inefficient but also results in significant bottlenecks in the diversity and responsiveness of the generated AI behavior.

[0003] Meanwhile, technologies such as deep reinforcement learning have made significant progress in the field of general artificial intelligence. Their ability to self-optimize through interaction with the environment and in a data-driven manner offers new possibilities for building more flexible and powerful game AI. However, directly applying end-to-end deep models to games presents many challenges: the model's decision-making process is like a "black box," difficult to understand and debug; the training process is unstable, requiring massive amounts of trial and error; and the generated strategies lack a clear logical structure, making it difficult to integrate with the game designer's high-level intentions. Summary of the Invention

[0004] This application provides a node-based data-driven decision-making method and system for game AI, which solves the technical problems of traditional game AI decision-making logic relying on manually written rules, making it difficult to adapt to complex dynamic environments, resulting in the inability to conduct end-to-end training and the difficulty to automatically optimize based on game feedback.

[0005] The first aspect of this application provides a node-based data-driven decision-making method for game AI, the method comprising: A neural node library for game AI is constructed, including neural perception nodes and neural action nodes. Based on the neural node library, a graph-like decision logic structure is assembled using graphical node connections. During game execution, based on the current game environment state data, the decision logic structure is traversed and executed to generate and execute AI control commands, while recording execution process data to obtain a differentiable decision trajectory. Performance reward values ​​are calculated based on the game result feedback generated after the execution of the AI ​​control commands. The internal network parameters of the neural nodes in the neural node library are updated using the performance reward values ​​and the differentiable decision trajectory to optimize the decision logic structure.

[0006] A second aspect of this application provides a node-based data-driven decision-making system for game AI, the system comprising: The system comprises the following modules: a node library construction module (building a neural node library for the game AI, including neural perception nodes and neural action nodes), a decision logic structure assembly module (assembling a graph-like decision logic structure based on the neural node library using graphical node connections), a decision logic structure execution module (traversing and executing the decision logic structure during game execution based on current game environment state data, generating and executing AI control commands, and recording execution process data to obtain a differentiable decision trajectory), a result feedback module (calculating performance reward values ​​based on game result feedback generated after the execution of AI control commands), and a decision logic structure optimization module (updating the internal network parameters of the neural nodes in the neural node library using the performance reward values ​​and the differentiable decision trajectory to optimize the decision logic structure).

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: First, a node library containing both sensory and action-related neural nodes is pre-built, and an editable graph-like decision structure is constructed using a visualized node connection method. During game execution, this decision structure is traversed and executed node by node according to the real-time environment state, outputting corresponding AI control commands. Simultaneously, the entire calculation process at each step is recorded as a differentiable trajectory that can be used for learning. Subsequently, performance rewards are calculated based on the results generated by these commands in the game. Combining the rewards with the differentiable trajectory, the neural network parameters within each node are trained and updated, thereby continuously improving and optimizing the overall decision-making logic, enabling the AI ​​to learn and improve through continuous feedback. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic diagram of a node-based data-driven decision-making method for game AI provided in an embodiment of this application.

[0010] Figure 2 This is a schematic diagram of the node-based data-driven decision-making system architecture for game AI provided in an embodiment of this application.

[0011] Figure labeling: Node library construction module 11, decision logic structure assembly module 12, decision logic structure execution module 13, result feedback module 14, decision logic structure optimization module 15. Detailed Implementation

[0012] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0013] Example 1, as Figure 1 As shown, this application provides a node-based data-driven decision-making method for game AI, the method including: Build a neural node library for game AI, including neural perception nodes and neural action nodes.

[0014] In this embodiment, during the system initialization phase, the environmental features, interaction methods, and set of executable actions of the target game are first analyzed. Based on the analysis results, several reusable neural node templates are established. Neural perception nodes are used to receive and process raw state information from the game environment, such as visual images, numerical attributes, terrain structures, character state parameters, and event trigger signals. Each perception node integrates a trainable forward inference sub-network, such as a convolutional neural network, a multilayer perceptron, or a Transformer encoder. By extracting features and making conditional judgments on the input state, it outputs a continuous condition satisfaction level for activation selection in subsequent decision logic. Neural action nodes are used to generate specific game operation instructions. Each action node contains an action type selection network and a parameter prediction sub-network. Based on the current state and the context of the decision path, it can output actions and their parameters that can be directly submitted to the game engine for execution, such as movement direction, skill release parameters, and resource allocation strategies. The action type selection network can be a multilayer perceptron, a long short-term memory network, or a Transformer encoder; the parameter prediction sub-network can be a multilayer perceptron, a convolutional neural network, or a Transformer encoder. All neural perception nodes and neural action nodes are stored as independent units in a neural node library. The node library supports expansion, registration, and version management, enabling different gameplay, roles, or task scenarios to quickly build new AI decision-making structures by selecting appropriate node combinations. Through this node library construction mechanism, the decision-making units are modularized, trainable, and reusable, providing a structural foundation for subsequent graph-based decision logic assembly and automated optimization.

[0015] Furthermore, the neural perception node is used to receive game environment state data and output continuous values ​​of condition satisfaction, and the neural action node is used to output executable actions and parameterized instructions.

[0016] Preferably, neural perception nodes are used to receive real-time state data from the game environment, analyze and process this data, and output a continuous value reflecting whether the current conditions are met. Specifically, each perception node integrates a trainable network structure that can encode and calculate the input multimodal game state information, such as visual frames, character values, enemy positions, and scene events, and generate a satisfaction level output representing the degree of condition matching through forward propagation. This condition satisfaction level can reflect simple logical judgments, such as "is the enemy approaching?", or express complex environmental understanding, such as "is the current terrain suitable for an ambush?" or "does the resource situation allow for the execution of a high-risk strategy?" Neural action nodes are used to generate specific executable action commands based on the triggering results of the decision-making process. Action nodes contain a network structure for action selection and parameter prediction. After receiving state features or output from higher-level nodes, they determine the type of action to be executed through forward inference and further generate the required action parameters, such as movement direction and speed, skill release coordinates, aiming angle, or resource allocation ratio. The generated actions and their parameters are output in the form of structured instructions, which can be directly submitted to the game control layer for execution, thereby driving the AI ​​character to produce corresponding behaviors in the game world. Through the above mechanism, neural perception nodes provide conditional judgment capabilities, and neural action nodes provide action decision-making capabilities; together, they constitute the basic execution unit for AI decision-making.

[0017] Furthermore, the neural perception node includes one or more of the following: visual perception sub-nodes, numerical state perception sub-nodes, and auditory / event perception sub-nodes.

[0018] Optionally, neural perception nodes can be composed of various types of perception sub-nodes to comprehensively understand the state information of the game environment from different dimensions. Among them, visual perception sub-nodes process image data from the game screen, integrating convolutional networks or visual Transformer structures to identify visual elements such as scene targets, obstacles, enemy and ally positions, skill effects, and terrain structures, generating visual semantic representations that can be used for subsequent decision-making. Numerical state perception sub-nodes read and analyze various game state parameters in numerical form, such as character health, resource quantity, cooldown time, speed values, and mission progress. They extract high-dimensional numerical state features through forward computation and output conditional matching degrees. Auditory / event perception sub-nodes capture non-visual environmental signals, such as game-triggered event messages, audio prompts, system broadcasts, teammate commands, and enemy skill warnings. These discrete signals are structured using event encoding networks (such as multilayer perceptrons) or sequence models (such as recurrent neural networks) to identify event categories and determine their impact on the current strategy. The three types of perception sub-nodes mentioned above can be used individually or combined to construct a comprehensive perception node, enabling the game AI to utilize visual, numerical, and event information simultaneously to form a more complete environmental perception result with stronger semantic understanding, thereby providing a reliable input basis for decision-making logic.

[0019] Furthermore, the neural perception nodes and the neural action nodes share a low-level feature extraction network based on game state encoding.

[0020] Optionally, neural perception nodes and neural action nodes share a low-level feature extraction network based on game state encoding to uniformly encode multi-source state data from the game environment. This low-level feature extraction network is typically composed of a deep neural network, including substructures such as image encoding branches, numerical state encoding branches, and event sequence encoding branches. It can extract features from visual frames, structured numerical information, and discrete event signals, respectively, and integrate the multimodal features into a stable and abstractly expressive shared state representation in the network's fusion layer. When performing forward inference, neural perception nodes directly call this shared feature representation as input, eliminating the need for each perception node to repeatedly construct low-level feature calculations, thereby improving recognition efficiency and ensuring the consistency of representations between different perception nodes. Similarly, when generating action types and parameters, neural action nodes also perform decision-making and inference based on this low-level feature representation, ensuring that the action output is closely related to the environmental state. Since both perception nodes and action nodes rely on the same set of state feature encoding networks, the parameters of this network can be updated through joint optimization during the training phase. This enables the network to simultaneously meet the needs of perception judgment and action decision-making, improve the overall coordination of decision-making and training efficiency, avoid redundant calculations and model redundancy, and achieve an efficient and unified foundation for state understanding and behavior generation.

[0021] Based on the aforementioned neural node library, a graph-like decision logic structure is assembled using graphical node connections.

[0022] In one embodiment, the system provides a visual node editing interface, allowing developers to select different types of neural perception nodes and neural action nodes from a neural node library, and graphically link these nodes according to the logical relationship of the decision-making process by dragging or connecting lines. Each node has a clearly defined input and output end, and can be connected hierarchically or in parallel according to the needs of condition judgment, state transition, or behavior chaining, thereby forming a graph-like decision logic structure containing branching, looping, and conditional triggering mechanisms. This decision logic structure not only supports the traditional top-down decision-making process of behavior tree, but can also construct complex graph networks, such as decision graphs with multiple root nodes, shared child nodes, or cross-layer references, to adapt to the strategic needs of different game scenarios. During the assembly process, the system automatically verifies the data type matching between nodes, the integrity of the execution path, and the legality of the loop structure, and generates an internal link description for each connection for computation to ensure that the graph structure can be traversed and forward-propagated in a predetermined logical order at runtime. Furthermore, the definition of the decision logic structure can be stored, versioned, and reused, allowing AI designers to adjust the node order, replace child nodes, or add strategy branches at any time. This enables complex AI logic to be constructed in a visual and modular way, providing a clear and editable structural framework for subsequent strategy execution and learning optimization.

[0023] During game operation, based on the current game environment state data, the decision logic structure is traversed and executed to generate and execute AI control commands, while recording execution process data to obtain a differentiable decision trajectory.

[0024] In one embodiment, during game execution, the system continuously receives current environmental state data and initiates a traversal execution process starting from the root node of the decision logic structure based on these real-time states. Specifically, in each frame or each decision cycle, the system sequentially visits the corresponding neural perception nodes and neural action nodes according to the connection relationships between nodes in the graph logic structure. When traversing to a perception node, its internal inference network is invoked, the environmental state features of the current frame are input, the condition satisfaction of the node is calculated, and the system determines whether to continue activating subsequent child node paths based on the node output and a preset threshold. When traversing to an action node, the system executes its internal parameter prediction subnetwork, generates corresponding AI control instructions and parameters, and submits the control instructions to the game execution layer, enabling the AI ​​character to perform actual actions in the game world. Throughout the traversal, the system synchronously records the input features, internal weights, activation outputs, and other information of each node, and concatenates these data in chronological order to construct a differentiable decision trajectory that can completely track each step of the calculation. This differentiable decision trajectory not only records the final output of the decision but also includes the intermediate network states used for backpropagation, enabling subsequent training processes to perform gradient calculations and parameter updates based on the real online decision process, achieving end-to-end differentiable optimization and policy self-improvement.

[0025] Furthermore, during game operation, based on the current game environment state data, the decision logic structure is traversed and executed to generate and execute AI control commands, including: In each game frame, a recursive traversal process is initiated starting from the root node of the decision logic structure. When the traversal reaches the neural perception node, the forward inference computation graph inside the neural perception node is invoked, the current game environment state data is input, and a confidence score is obtained. The confidence score is compared with a preset dynamic threshold, and the current node and its subsequent connected child nodes are activated based on the comparison result. When the traversal reaches the neural action node, the forward propagation of the parameter prediction subnetwork inside the neural action node is executed to generate AI control instructions and parameters, which are then submitted to the game execution layer.

[0026] Preferably, in each game frame, the system first initiates a complete recursive traversal process starting from the root node of the decision logic structure. This traversal proceeds downwards according to the connections between nodes, triggering corresponding execution logic when encountering different types of nodes. When traversing to a neural perception node, the game environment state data collected in the current frame is input into the forward inference computation graph inside that node. This forward inference computation graph performs feature calculations on the input data based on the included network model, outputting a satisfaction score representing the degree to which the condition is satisfied, as a continuous confidence score. Subsequently, this confidence score is compared with a dynamic threshold set inside the node, and the node is activated based on the comparison result. If the confidence score exceeds the threshold, the current condition is considered satisfied, and the recursive traversal continues, activating one or more child nodes connected to the node. If the confidence score does not reach the threshold, the relevant branches are skipped to reduce the overhead of invalid decision paths. When the traversal reaches the neural action node, it switches to the action generation process. The parameter prediction subnetwork within the action node (composed of the previously constructed network structure for action prediction) performs forward propagation. This parameter prediction subnetwork predicts the specific action type and its associated parameters, such as movement direction and speed, skill release coordinates, and attack target, based on shared state feature representations and activation information from upstream nodes. The generated action instructions are encapsulated into structured AI control instructions and submitted to the game execution layer in real time, where the game engine drives the AI ​​character to complete the corresponding behavior. By recursively traversing the entire decision logic structure in each game frame, the system can continuously generate real-time intelligent behaviors in response to the current environmental state.

[0027] Furthermore, by simultaneously recording execution process data, a differentiable decision trajectory is obtained, including: During the execution of the decision logic structure, a computational graph shadow is constructed synchronously. The computational graph shadow runs in parallel with the game execution logic and is used to completely record the input data, internal weights, activation outputs, and connection relationships between nodes during the forward propagation of each neural node. The computational graph shadows of each decision step are linked in time sequence to generate a decision trajectory data chain. The decision trajectory data chain is aligned and stored with the original game state frames with corresponding timestamps to generate the differentiable decision trajectory.

[0028] Optionally, throughout the execution of the decision logic structure, the system will simultaneously initiate a computational graph shadow construction mechanism parallel to the main decision-making process, without affecting the efficiency of real-time game operation. Specifically, when each neural node in the decision logic structure performs forward propagation in the main process, the system will synchronously create a corresponding node record unit in the shadow computational graph, and write the input data used by the node for this calculation, the network weight snapshot inside the node, the activation output result, and the connection relationship between the node and its upstream and downstream nodes into the record unit. The shadow computational graph does not participate in game behavior control, but is only used to reconstruct the internal computation process of decision reasoning. Subsequently, after each decision step (usually corresponding to a frame or a decision cycle), the shadow computational graph generated by the step is encapsulated as an independent segment, and multiple segments are linked sequentially according to time order to form a decision trajectory data chain that can continuously track the entire decision-making process of the AI. This decision trajectory data chain not only saves the computation process of each node, but also retains the activation path and branch selection between nodes, so that the evolution of the decision logic in the temporal dimension can be completely reproduced. Finally, this decision trajectory data chain is aligned and stored with the original state frames recorded by the game engine according to the same timestamp to form a complete differentiable decision trajectory. This differentiable decision trajectory can simultaneously associate the calculation process with the real environment state, so that when backpropagation is performed in the training phase, the environmental feedback can be accurately associated with the corresponding neural node calculation process to achieve end-to-end gradient optimization.

[0029] The performance bonus value is calculated based on the game result feedback generated after the execution of the AI ​​control command.

[0030] In one embodiment, after the AI ​​performs an action, its impact on the game situation is monitored in real time. Game outcome feedback data triggered by the action is obtained from the game engine or combat logic, such as changes in character health, resource increases / decreases, mission progress, enemy status changes, changes in positional advantages, and team benefits—multi-dimensional indicators. The system predefines a reward evaluation mechanism in the background to measure the quality of AI behavior. This includes both ultimate rewards based on the final outcome of the game or the completion of the objective, and procedural rewards reflecting the quality of intermediate behaviors. After collecting action feedback, the system quantifies this feedback according to reward rules, transforming it into numerical reward signals that can be used for learning. Finally, the various reward signals are weighted together to generate a performance reward value that reflects the overall performance of the current action. This provides a quantitative basis for subsequent updates to neural node parameters and optimization of decision logic, enabling the AI ​​to gradually learn better behavioral strategies.

[0031] Furthermore, based on the game result feedback generated after the execution of the AI ​​control commands, a performance reward value is calculated, including: Define a sparse ultimate reward function based on the game's ultimate goal; introduce a reward allocation network, taking a differentiable decision trajectory as input, and based on the sparse ultimate reward function, inversely allocate the sparse ultimate reward to each neural node in the decision trajectory to obtain the allocated reward; define a dense sub-reward function related to intermediate behaviors to evaluate the performance of the game AI in movement efficiency, resource management, and tactical execution, and weight the basic sub-reward obtained based on the dense sub-reward function with the allocated reward from the reward allocation network to obtain the performance reward value.

[0032] Preferably, in the performance reward calculation process, a sparse ultimate reward function is first constructed based on the final win / loss conditions of the game or the task completion goal. This sparse ultimate reward function typically only generates reward signals when the game ends or a key objective is achieved, such as "win +1, lose -1" or "task completion +R, task failure 0". Its characteristic is that the reward appears infrequently but is highly representative. Since this reward only appears at the end of the decision sequence, it cannot be directly used to guide the learning of each node in the entire decision chain. Therefore, the system further introduces a reward allocation network. This reward allocation network takes the aforementioned differentiable decision trajectory as input, reads the node activation, forward output, behavior selection, and path information between nodes at each step of the trajectory, and propagates the final reward backward along the time dimension through the sparse ultimate reward function. The sparse reward that originally only appeared at the end is reasonably decomposed and allocated to each neural node in the trajectory, thereby obtaining the allocated reward corresponding to each node, enabling the system to know which nodes play a key role in the final result. To compensate for the learning lag caused by the sparse ultimate reward, the system also defines a set of dense sub-reward functions to evaluate the quality of the AI's real-time strategies during game execution, such as the efficiency of movement paths, the rationality of resource allocation, the precision of combat operations, and the timeliness of risk avoidance. These sub-reward functions can be calculated frame-by-frame based on real-time data output by the game engine, providing each node with a basic sub-reward reflecting the quality of local decisions. Finally, the system weights and fuses the dense sub-rewards with the allocated rewards generated by the reward allocation network according to set weights to obtain a performance reward value representing the overall performance of the current decision step. This performance reward value retains endgame orientation while containing stable and continuous intermediate supervision signals, making the training process more efficient and stable, thus providing a precise learning target for subsequent neural node optimization.

[0033] Using the performance reward value and the differentiable decision trajectory, the internal network parameters of the neural nodes in the neural node library are updated to optimize the decision logic structure.

[0034] In one embodiment, the system continuously collects differentiable decision trajectories and their corresponding performance reward values ​​during execution, and inputs them as training samples into the experience replay buffer of the neural nodes. Since the differentiable decision trajectories completely record the input features, network weights, activation results, and inter-node connections of each neural node at each forward propagation step, the system can construct a complete backpropagation path based on these records. Subsequently, using the performance reward value as the optimization objective, the gradient contribution of each node in the trajectory is calculated, and backpropagation is performed along the shadow computation graph to determine the adjustment direction and magnitude of the internal parameters of each neural node. Through gradient descent, policy gradient, or other reinforcement learning optimization algorithms, the network weights of the corresponding nodes in the neural node library are updated in real time or periodically, enabling the nodes to output perceptual results or action commands that better align with the reward objective in similar environments. After the parameter updates are completed, the capabilities of the nodes in the neural node library are enhanced, and the system automatically applies the updated nodes back to the existing decision logic structure. Policy upgrades can be completed without reassembling the graph structure, achieving a sustainable, interpretable, and differentiable intelligent decision optimization process.

[0035] Furthermore, by using the performance reward value and the differentiable decision trajectory to update the internal network parameters of the neural nodes in the neural node library, the decision logic structure is optimized, including: Periodically store the performance reward values ​​and corresponding differentiable decision trajectories collected during game operation into the experience replay buffer; on a dedicated training server, sample batch trajectory data from the experience replay buffer, adopt a distributed training architecture, and multiple training ends perform backpropagation and calculate gradients in parallel through differentiable decision trajectories, and a parameter server performs gradient aggregation and asynchronous updates; optimize the decision logic structure with the updated neural node network parameters.

[0036] Preferably, during the actual gameplay phase, the performance reward values ​​and corresponding differentiable decision trajectories generated by AI are continuously collected. These data entries are periodically cached in an experience replay buffer in chronological order. This experience replay buffer is managed using a queue or circular storage structure, enabling long-term storage of diverse decision samples from multiple rounds of gameplay, providing a rich and stable data source for subsequent training. Subsequently, offline or quasi-online learning tasks are initiated on a dedicated training server, and trajectory data is randomly sampled in batches from the experience replay buffer to form training batch trajectory data. This training batch trajectory data is then distributed to multiple distributed training endpoints. Each training endpoint independently executes a complete backpropagation process based on the node inputs, activation outputs, and connection relationships recorded in the trajectory, calculating the gradient information for the corresponding batch of samples. A parameter server is set up in the distributed training endpoints to receive the gradient data uploaded by all training endpoints, perform aggregation operations such as averaging and weighted fusion, and, after aggregation, uniformly update the global neural network parameters asynchronously. This update is automatically synchronized to all neural perception nodes and neural action nodes in the node library, giving them new parameter weights. After the parameters are updated, these optimized neural nodes are immediately reloaded into the decision logic structure of the online operating environment, enabling the game AI to use better perception capabilities and action generation strategies in subsequent actual games. Through this "sampling-training-update-application" loop mechanism, continuous iterative closed-loop optimization is achieved, allowing the game AI to continuously improve its strategies, enhance decision-making quality, and improve environmental adaptability through the accumulation of a large amount of game experience.

[0037] Furthermore, optimizing the decision logic structure also includes: The system continuously monitors the usage frequency and average reward contribution of each node in the decision logic structure. When the average contribution of any node is lower than a preset threshold for a duration longer than a preset duration, the corresponding node is marked as a node to be optimized. For the node to be optimized, a structure search process is triggered, a set of candidate substructures are selected from the candidate neural node library, and a simulation evaluation is performed on a validation set composed of historical data. The candidate substructure with the best evaluation performance is used to replace the node to be optimized.

[0038] Preferably, during actual operation, the system continuously monitors all neural nodes within the decision logic structure, collecting in real-time the activation frequency of each node during the game and the average contribution value of the node calculated from performance rewards. The system uses a sliding window or periodically summarizes these statistical indicators to ensure the stability and reliability of node evaluation. When it is detected that the average contribution value of a node is consistently lower than the set performance threshold, and the duration of the low contribution state exceeds a predefined time period, such as a fixed time window, it is determined that the node can no longer play an effective role in the current strategy structure. At this time, the system automatically marks the node as a node to be optimized and records its position in the decision logic structure and its dependent upstream nodes. After the node is marked, the system initiates a structure search process, selecting a set of candidate substructures related to the function of the node from a pre-built candidate neural node library. Candidate substructures may include different forms of perception node combinations, improved action node structures, etc. The system inserts each candidate substructure into the original node position in the decision logic and uses a validation set composed of historical running data to simulate and reason about the candidate structures, monitoring their average reward contribution. Ultimately, the system selects the candidate substructure with the optimal average reward contribution on the validation set and replaces the original low-contribution node with it. This allows the entire decision logic structure to achieve structural optimization while maintaining its original coherence. Through this automated structure search and replacement mechanism, the system can not only optimize node parameters but also dynamically adjust the decision structure itself, enabling the AI ​​strategy structure to continuously evolve and become more adaptable and capable of long-term performance improvement.

[0039] In summary, the embodiments of this application have at least the following technical effects: First, a neural node library for the game AI is constructed, including neural perception nodes and neural action nodes. Next, based on this neural node library, a graph-like decision logic structure is assembled using graphical node connections. Then, during game execution, based on the current game environment state data, the decision logic structure is traversed and executed to generate and execute AI control commands, while simultaneously recording execution process data to obtain a differentiable decision trajectory. Then, based on the game result feedback generated after the execution of the AI ​​control commands, a performance reward value is calculated. Finally, using the performance reward value and the differentiable decision trajectory, the internal network parameters of the neural nodes in the neural node library are updated to optimize the decision logic structure. This solves the technical problems of traditional game AI decision logic relying on manually written rules, making it difficult to adapt to complex dynamic environments, resulting in the inability to perform end-to-end training and automatic optimization based on game feedback. It achieves the technical effect of constructing a flexible and adjustable decision logic structure through graphical node assembly, and combining differentiable decision trajectories and performance reward feedback to achieve end-to-end gradient optimization, thereby improving the adaptability, intelligence, and optimization efficiency of game AI decision-making.

[0040] Example 2, based on the same inventive concept as the node-based data-driven decision-making method for game AI in the foregoing examples, such as... Figure 2 As shown, this application provides a node-based data-driven decision-making system for game AI, the system comprising: Node library construction module 11: Constructs a neural node library for the game AI, including neural perception nodes and neural action nodes; Decision logic structure assembly module 12: Based on the neural node library, assembles a graphical decision logic structure using a graphical node connection method; Decision logic structure execution module 13: During game operation, based on the current game environment state data, traverses and executes the decision logic structure, generates and executes AI control commands, and records execution process data to obtain a differentiable decision trajectory; Result feedback module 14: Calculates the performance reward value based on the game result feedback generated after the execution of the AI ​​control commands; Decision logic structure optimization module 15: Utilizes the performance reward value and the differentiable decision trajectory to update the internal network parameters of the neural nodes in the neural node library, thereby optimizing the decision logic structure.

[0041] Furthermore, the node library construction module 11 is used to perform the following methods: The neural perception node is used to receive game environment state data and output continuous values ​​of condition satisfaction, while the neural action node is used to output executable actions and parameterized instructions.

[0042] Furthermore, the node library construction module 11 is used to perform the following methods: The neural perception nodes include one or more of the following: visual perception sub-nodes, numerical state perception sub-nodes, and auditory / event perception sub-nodes.

[0043] Furthermore, the node library construction module 11 is used to perform the following methods: The neural perception nodes and the neural action nodes share a low-level feature extraction network based on game state encoding.

[0044] Furthermore, the decision logic structure execution module 13 is used to execute the following methods: In each game frame, a recursive traversal process is initiated starting from the root node of the decision logic structure. When the traversal reaches the neural perception node, the forward inference computation graph inside the neural perception node is invoked, the current game environment state data is input, and a confidence score is obtained. The confidence score is compared with a preset dynamic threshold, and the current node and its subsequent connected child nodes are activated based on the comparison result. When the traversal reaches the neural action node, the forward propagation of the parameter prediction subnetwork inside the neural action node is executed to generate AI control instructions and parameters, which are then submitted to the game execution layer.

[0045] Furthermore, the decision logic structure execution module 13 is used to execute the following methods: During the execution of the decision logic structure, a computational graph shadow is constructed synchronously. The computational graph shadow runs in parallel with the game execution logic and is used to completely record the input data, internal weights, activation outputs, and connection relationships between nodes during the forward propagation of each neural node. The computational graph shadows of each decision step are linked in time sequence to generate a decision trajectory data chain. The decision trajectory data chain is aligned and stored with the original game state frames with corresponding timestamps to generate the differentiable decision trajectory.

[0046] Furthermore, the result feedback module 14 is used to perform the following method: Define a sparse ultimate reward function based on the game's ultimate goal; introduce a reward allocation network, taking a differentiable decision trajectory as input, and based on the sparse ultimate reward function, inversely allocate the sparse ultimate reward to each neural node in the decision trajectory to obtain the allocated reward; define a dense sub-reward function related to intermediate behaviors to evaluate the performance of the game AI in movement efficiency, resource management, and tactical execution, and weight the basic sub-reward obtained based on the dense sub-reward function with the allocated reward from the reward allocation network to obtain the performance reward value.

[0047] Furthermore, the decision logic structure optimization module 15 is used to execute the following method: Periodically store the performance reward values ​​and corresponding differentiable decision trajectories collected during game operation into the experience replay buffer; on a dedicated training server, sample batch trajectory data from the experience replay buffer, adopt a distributed training architecture, and multiple training ends perform backpropagation and calculate gradients in parallel through differentiable decision trajectories, and a parameter server performs gradient aggregation and asynchronous updates; optimize the decision logic structure with the updated neural node network parameters.

[0048] Furthermore, the decision logic structure optimization module 15 is used to execute the following method: The system continuously monitors the usage frequency and average reward contribution of each node in the decision logic structure. When the average contribution of any node is lower than a preset threshold for a duration longer than a preset duration, the corresponding node is marked as a node to be optimized. For the node to be optimized, a structure search process is triggered, a set of candidate substructures are selected from the candidate neural node library, and a simulation evaluation is performed on a validation set composed of historical data. The candidate substructure with the best evaluation performance is used to replace the node to be optimized.

[0049] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A node-based data-driven decision-making method for game AI, characterized in that, include: Build a neural node library for game AI, including neural perception nodes and neural action nodes; Based on the neural node library, a graph-like decision logic structure is assembled using a graphical node connection method. During game operation, based on the current game environment state data, the decision logic structure is traversed and executed to generate and execute AI control commands, while recording execution process data to obtain a differentiable decision trajectory; Calculate the performance bonus value based on the game result feedback generated after the execution of the AI ​​control commands; Using the performance reward value and the differentiable decision trajectory, the internal network parameters of the neural nodes in the neural node library are updated to optimize the decision logic structure.

2. The node-based data-driven decision-making method for game AI as described in claim 1, characterized in that, During game operation, based on the current game environment state data, the decision logic structure is traversed and executed to generate and execute AI control commands, including: In each game frame, a recursive traversal process is initiated starting from the root node of the decision logic structure. When the traversal reaches the neural perception node, the forward inference computation graph inside the neural perception node is called, the current game environment state data is input, and the confidence score is obtained. The confidence score is compared with a preset dynamic threshold, and the comparison result is used to determine whether to activate the current node and its subsequent connected child nodes. When traversing to a neural action node, the forward propagation of the parameter prediction subnetwork within the neural action node is performed to generate AI control commands and parameters, which are then submitted to the game execution layer.

3. The node-based data-driven decision-making method for game AI as described in claim 2, characterized in that, Simultaneously, execution process data is recorded to obtain a differentiable decision trajectory, including: During the execution of the decision logic structure, a computation graph shadow is constructed synchronously. The computation graph shadow runs in parallel with the game execution logic and is used to fully record the input data, internal weights, activation outputs, and connection relationships between nodes during the forward propagation of each neural node. Link the computational shadow graphs of each decision-making step according to the time series to generate a decision trajectory data chain; The decision trajectory data chain is aligned and stored with the original game state frame with the corresponding timestamp to generate the differentiable decision trajectory.

4. The node-based data-driven decision-making method for game AI as described in claim 1, characterized in that, Based on the game result feedback generated after the execution of the AI ​​control commands, the performance bonus value is calculated, including: Define a sparse ultimate reward function based on the game's ultimate goal; A reward allocation network is introduced, with a differentiable decision trajectory as input. Based on the sparse ultimate reward function, the sparse ultimate reward is inversely allocated to each neural node in the decision trajectory to obtain the allocated reward. A dense sub-reward function related to intermediate behaviors is defined to evaluate the performance of the game AI in movement efficiency, resource management and tactical execution. The performance reward value is obtained by weighting the basic sub-reward obtained based on the dense sub-reward function with the allocated reward from the reward allocation network.

5. The node-based data-driven decision-making method for game AI as described in claim 1, characterized in that, Using the performance reward value and the differentiable decision trajectory, the internal network parameters of the neural nodes in the neural node library are updated to optimize the decision logic structure, including: Periodically store the performance reward values ​​collected during game operation and the corresponding differentiable decision trajectories into the experience replay buffer; On a dedicated training server, batch trajectory data is sampled from the experience replay buffer. A distributed training architecture is adopted, with multiple training ends performing backpropagation and calculating gradients in parallel through differentiable decision trajectories. Gradient aggregation and asynchronous updates are performed by a parameter server. The decision logic structure is optimized using the updated neural node network parameters.

6. The node-based data-driven decision-making method for game AI as described in claim 1, characterized in that, Optimizing the decision logic structure further includes: Continuously monitor the usage frequency and average reward contribution of each node in the decision-making logic structure; If the duration for which the average contribution of any node is below a preset threshold is longer than a preset duration, the corresponding node will be marked as a node to be optimized. The structure search process is triggered for the node to be optimized. A set of candidate substructures is selected from the candidate neural node library, and a simulation evaluation is performed on a validation set composed of historical data. The candidate substructure with the best evaluation performance replaces the node to be optimized.

7. The node-based data-driven decision-making method for game AI as described in claim 1, characterized in that, The neural perception node is used to receive game environment state data and output continuous values ​​of condition satisfaction, while the neural action node is used to output executable actions and parameterized instructions.

8. The node-based data-driven decision-making method for game AI as described in claim 7, characterized in that, The neural perception nodes include one or more of the following: visual perception sub-nodes, numerical state perception sub-nodes, and auditory / event perception sub-nodes.

9. The node-based data-driven decision-making method for game AI as described in claim 7, characterized in that, The neural perception nodes and the neural action nodes share a low-level feature extraction network based on game state encoding.

10. A node-based data-driven decision-making system for game AI, characterized in that: A node-based data-driven decision-making method for game AI as described in any one of claims 1-9, comprising: Node library building module: Builds the neural node library for game AI, including neural perception nodes and neural action nodes; Decision logic structure assembly module: Based on the neural node library, it assembles a graph-like decision logic structure by connecting nodes in a graphical manner; Decision logic structure execution module: During game operation, based on the current game environment state data, the decision logic structure is traversed and executed to generate and execute AI control commands, while recording execution process data to obtain a differentiable decision trajectory; Result Feedback Module: Calculates performance reward value based on the game result feedback generated after the execution of the AI ​​control command; Decision logic structure optimization module: Utilizes the performance reward value and the differentiable decision trajectory to update the internal network parameters of the neural nodes in the neural node library, thereby optimizing the decision logic structure.

Images