Wargame deduction intelligent game decision-making method and system

Abstract

The invention discloses a war game deduction intelligent game decision-making method and a war game deduction intelligent game decision-making system. The method comprises the following steps: S1, acquiring a current battlefield situation, constructing a root node of a game tree, and enumerating all legal actions; s2, taking each legal action as a child node of a root node, creating an independent parallel search task for each child node, and distributing and executing the task through a work stealing thread pool; s3, independently executing Monte Carlo tree search iteration for multiple times by each parallel search task; and S4, when a termination condition is met, summarizing statistical results of all the parallel search tasks, and selecting an action corresponding to the child node with the maximum access frequency under the root node as a final decision instruction. According to the intelligent game algorithm, a concurrent mode is combined with an atomic variable lock-free mechanism, and the problems that traditional Monte Carlo tree search is low in search efficiency, serious in thread competition and large in memory overhead in a multi-thread environment are solved.

Description

Technical Field

[0001] This invention relates to the field of wargaming technology, and in particular to a wargaming intelligent game decision-making method and system. Background Technology

[0002] In existing wargaming and intelligent game systems, situation assessment mainly faces the following technical bottlenecks: 1. Limitations of rule-based systems: Traditional methods rely on expert-preset rule bases (such as the Drools engine) and employ static if-then logic. This approach lacks adaptability, cannot cope with dynamically changing and uncertain factors on the battlefield, and struggles to construct adversarial intelligent game behavior.

[0003] 2. Performance Bottlenecks of Traditional Single-Threaded MCTS: While Monte Carlo Tree Search (MCTS) possesses intelligent exploration capabilities, the state space complexity of wargames far exceeds that of Go (larger branching factors, longer game length). In a Java single-threaded execution environment, limited by CPU clock speed, the number of simulations per unit time is extremely low, resulting in slow convergence speed, which cannot meet the real-time response requirements of wargames at the second or even millisecond level. 3. Multi-threaded concurrency issues: If multi-threaded parallel search of the shared game tree is directly adopted, multiple threads accessing the same node simultaneously will lead to severe lock contention, reducing throughput. Without control, multiple threads tend to focus on searching the same high-value path, resulting in insufficient search diversity and getting trapped in local optima. Therefore, there is an urgent need for an intelligent game theory method that can fully utilize the computing power of multi-core CPUs, ensure data consistency through lock-free concurrency mechanisms, and introduce dynamic load balancing strategies to maximize the breadth of search. Summary of the Invention

[0004] In view of this, the purpose of this invention is to propose an intelligent game decision-making method for wargaming, which uses a concurrent approach combined with an atomic variable lock-free mechanism to solve the problems of low search efficiency, severe thread contention and high memory overhead of traditional Monte Carlo tree search in multi-threaded environments.

[0005] According to one aspect of the present invention, a wargaming intelligent game decision-making method is provided, comprising the following steps: S1. Obtain the current battlefield situation, construct the root node of the game tree, and enumerate all legal actions; S2. Treat each legal action as a child node of the root node, and create an independent parallel search task for each child node, which is then distributed for execution through a work-stealing thread pool. S3. Each parallel search task independently executes multiple Monte Carlo tree search iterations, each iteration including: Selection phase: Starting from the root node, traverse downwards to the leaf node according to the node selection strategy. During the traversal, apply virtual loss deduction to the access count of the currently visited node through atomic operations to temporarily reduce the probability that the node will be selected by other threads. Expansion phase: If a leaf node is not fully expanded and is not the final stage, add a new child node to it; Simulation Phase: Starting from the current node, the simulation proceeds to the endgame or maximum depth using a heuristic random strategy to obtain reward values; Backtracking phase: From bottom to top along the search path, perform virtual loss cancellation, access count accumulation and reward value accumulation on each node on the path through atomic operations to achieve lock-free update; S4. When the termination condition is met, summarize the statistical results of all parallel search tasks and select the action corresponding to the child node with the largest number of visits under the root node as the final decision instruction.

[0006] In the above technical solution, the game tree search task is decomposed into independent parallel units corresponding one-to-one with legal actions, and dynamic load balancing in a multi-core environment is achieved through a work-stealing thread pool. In each Monte Carlo tree search iteration, this method, based on the standard four-stage process (selection, expansion, simulation, and backtracking), introduces a virtual loss deduction mechanism based on atomic operations in the selection stage, and uses atomic operations to achieve lock-free statistical updates in the backtracking stage. This method constructs a complete technical closed loop from situation input to decision output, aiming to solve the problems of insufficient real-time performance and concurrency conflicts faced by traditional Monte Carlo tree search in large-scale state-space wargame scenarios.

[0007] The key feature of this scheme is that by deeply integrating the concurrency control mechanism with the iterative process of Monte Carlo tree search, it achieves load balancing and data consistency for high-concurrency search without introducing a global lock.

[0008] First, this method encapsulates each legitimate action as an independent parallel search task and distributes it for execution using a work-stealing thread pool. This design differs from traditional single-threaded search or simple multi-threaded shared tree structures; its core lies in refining the task granularity to the action level, enabling the thread pool to dynamically migrate tasks based on the actual load status of each thread. The work-stealing algorithm ensures that when some threads become idle early due to differences in the complexity of search branches, they can proactively steal tasks from the tail of the busy thread queue, effectively avoiding idle computing power caused by an unbalanced search tree. The resulting technical effect is a substantial increase in the number of Monte Carlo tree search iterations per unit time in a multi-core processor environment, providing a computing power foundation for real-time decision-making in wargaming scenarios.

[0009] Secondly, this method introduces a virtual loss mechanism during the selection phase and temporarily deducts the node access count through atomic operations. Specifically, when a thread accesses a node, an atomic subtraction is immediately performed to reduce the node's access count by a preset virtual loss value, causing the node's weight to decrease instantaneously in the node selection calculations of other threads. The essence of this mechanism is to construct an implicit load balancing strategy—without a central scheduler or global lock, it guides concurrent threads to automatically distribute to branches with lower current access frequency simply by dynamically changing the statistical values ​​of local nodes. Compared to traditional parallel Monte Carlo tree search methods that rely on lock mechanisms to control thread access or allow threads to concentrate on searching the same high-value path, this mechanism eliminates the blocking overhead caused by lock contention and maintains search diversity through the "temporary" nature of the virtual loss, preventing the algorithm from prematurely converging to a local optimum.

[0010] Finally, this method employs atomic operations during the backtracking phase to perform a composite update of virtual loss offsetting, access count accumulation, and reward value accumulation. Node statistics (access count, cumulative reward) are stored as atomic variables, and during backtracking, a single atomic addition simultaneously restores the virtual loss value and accumulates the actual access count. The essence of this design lies in applying lock-free programming principles throughout the entire lifecycle of the search path—from the application of virtual loss in the selection phase to the statistical recovery in the backtracking phase. The entire process requires no locking of nodes, allowing different threads to update different nodes and even different statistical fields of the same node in parallel. The resulting technical benefits are that, in high-concurrency scenarios, mutual blocking between threads is completely eliminated, ensuring system throughput and response stability. Furthermore, due to the non-blocking nature of atomic operations, the risk of priority inversion or deadlock caused by lock contention is avoided.

[0011] In summary, the essence of this technology is to construct a high-concurrency Monte Carlo tree search framework suitable for wargame scenarios through task-level parallel decomposition, implicit load balancing guided by virtual loss, and lock-free backtracking based on atomic operations. Its beneficial effects are: significantly improving parallel search efficiency while ensuring search diversity and decision quality; eliminating lock contention overhead in multi-threaded environments; and ensuring statistical consistency and real-time system response capabilities during high-concurrency iteration.

[0012] The innovation of this invention lies not only in parallelizing Monte Carlo tree search, but also in systematically solving the two key problems of load balancing and concurrency control in parallel search through the combination of virtual loss mechanism and lock-free backtracking. This feature, together with other mechanisms such as state object reuse and timeout circuit breaking in the overall technical solution, jointly supports an intelligent game theory method that can stably operate and make real-time decisions in highly complex wargaming scenarios, providing a core technical foundation for achieving adaptive interaction between adversarial agents and complex battlefield environments.

[0013] In some embodiments, the virtual loss deduction and offset are specifically as follows: the node access count is stored in atomic integer type, and the cumulative reward is stored in atomic double-precision floating-point type; when a thread enters a node, an atomic subtraction operation is performed to deduct a preset virtual loss value; when a thread leaves a node, an atomic addition operation is performed to add back the virtual loss value and add an additional real access count; wherein, the virtual loss value is a configurable positive integer, and its value range dynamically increases with the increase of the total number of concurrent threads.

[0014] The aforementioned technical solution specifically defines the complete implementation scheme of the virtual loss mechanism, clarifying the storage method of the game tree node statistics (access counts use atomic integer storage, and cumulative rewards use atomic double-precision floating-point storage), the atomic operation process of virtual loss deduction and offset (when a thread enters a node, an atomic subtraction operation is performed to deduct the virtual loss value; when leaving a node, an atomic addition operation is performed to add back the virtual loss value and additionally accumulate the real access count), and the configuration rules for the virtual loss value (configurable as a positive integer, with the value range dynamically increasing as the total number of concurrent threads increases). This feature determines the concurrent access correctness of the game tree node statistics in a multi-threaded environment, the load balancing effect of search behavior, and the concurrent throughput capacity of the system.

[0015] The essential feature of this technology is that by storing all node statistics (access count and cumulative reward) in atomic type and combining it with a dynamic adjustment mechanism for virtual loss value, a completely lock-free node access control scheme with adaptive concurrency is constructed.

[0016] First, this feature explicitly states that node access counts are stored using atomic integers, while accumulated rewards are stored using atomic double-precision floating-point numbers. This storage scheme differs from traditional implementations that use ordinary integers and floating-point numbers in conjunction with external locks (such as `synchronized`, a Java mutex synchronization primitive implemented based on the JVM's built-in monitor lock, which forces threads to acquire ownership of the object monitor before entering a synchronized block, thus ensuring atomicity and visibility in memory semantics. Its underlying implementation relies on lock flags in the object header, supporting lock escalation mechanisms from biased locking and lightweight locking to heavyweight locking to ensure thread safety for accessing shared resources in a multi-threaded concurrent environment) or simply atomicating access counts while still relying on lock protection for accumulated rewards. Its core design ensures thread safety by delegating all update operations of node statistics to the atomic types themselves, allowing the atomic decrement operation (deducting the virtual loss value) performed when a thread enters a node and the atomic increment operation (adding back the virtual loss value and accumulating the actual access count once more) performed when a thread leaves a node to complete without blocking other threads. Because atomic operations are implemented using hardware-level CAS (Compare-And-Swap) instructions, their execution does not involve thread scheduling or context switching at the operating system level. Unlike implementations that rely on lock mechanisms for critical section protection, this feature completely eliminates the overhead of thread blocking, waiting, and waking up during node access, refining the granularity of concurrency control from the node level to the single-count update level. The resulting technical effect is that, in high-concurrency scenarios, dozens or even hundreds of threads can simultaneously access different nodes of the same game tree, and even simultaneously update the access count and cumulative reward of the same node (ensuring eventual consistency through the CAS mechanism), substantially improving the system's throughput and response stability.

[0017] Secondly, the feature of "performing an atomic subtraction operation when a thread enters a node, deducting a preset virtual loss value; and performing an atomic addition operation when a thread leaves a node, adding back the virtual loss value and additionally accumulating the real access count" constitutes a complete paired execution logic. The thread subtracts the virtual loss at the node access entry point and adds back the virtual loss and additionally increases the real access count at the backtracking exit point. This paired design ensures the final accuracy of node statistics after the complete iteration cycle. Compared to the traditional mode of locking before updating and unlocking before other threads operate, this design limits the existence of virtual loss to the time window of a single-threaded node access. This achieves temporary access heat adjustment (by instantly reducing the node's attractiveness to other threads through atomic subtraction) and avoids the permanent impact of virtual loss values ​​on node statistics through atomic addition during backtracking, ensuring the predictability of the search algorithm's convergence behavior. Simultaneously, the design of simultaneously completing virtual loss recovery and real access accumulation in a single atomic addition reduces the number of atomic operation calls and lowers the performance overhead caused by memory barriers.

[0018] Furthermore, the design of "the virtual loss value being a configurable positive integer, with its range dynamically increasing as the total number of concurrent threads increases" in this feature endows the virtual loss mechanism with adaptive adjustment capabilities. The core function of the virtual loss value is to temporarily reduce the attractiveness of a node to other threads, and its effect must be matched with the level of concurrency: the more concurrent threads there are, the higher the probability that a node will be accessed by multiple threads simultaneously. If a sufficiently large virtual loss is not applied, the thread splitting effect will be weakened due to the "cancellation effect" (when multiple threads simultaneously deduct from the same node, the cumulative effect of their virtual losses will weaken each other); conversely, if a large virtual loss is used when the number of threads is small, it may lead to overexploration, weakening the utilization efficiency of high-value paths. This feature achieves a positive correlation between the virtual loss value and the number of concurrent threads through a dynamic adjustment mechanism, enabling the load balancing effect to adapt to different operating environments. This adjustment, combined with atomic operations, allows the virtual loss mechanism to maintain lock-free characteristics while possessing environmental adaptability, distinguishing it from traditional parallel Monte Carlo tree search implementations with fixed parameters.

[0019] Finally, the use of atomic double-precision floating-point storage for the cumulative reward in this feature has independent technical significance. In parallel Monte Carlo tree search, the update frequency of the cumulative reward is the same as that of the access count. If the cumulative reward is protected by ordinary floating-point data and a locking mechanism, even if the access count is updated without locks, threads still need to compete for the lock at the cumulative reward update point, forming a performance bottleneck. This feature also incorporates the cumulative reward into atomic operation management, enabling the composite update in the backtracking phase (virtual loss cancellation + real access accumulation + reward accumulation) to be completed entirely in a lock-free state, achieving completely lock-free concurrent access to node statistics.

[0020] In summary, the essence of this technology is to construct a completely lock-free, adaptive virtual loss control mechanism by combining atomic type storage with dynamic parameter adjustment. Its beneficial effects are as follows: during multi-threaded parallel search, mutual blocking between threads is completely eliminated, and the throughput of node statistics updates is increased by orders of magnitude; the virtual loss value dynamically changes with the level of concurrency, ensuring optimized load balancing under different operating conditions; the paired execution of atomic subtraction and atomic addition ensures the consistency of node statistics, providing a reliable data foundation for the correct calculation of subsequent node selection strategies; and the atomic storage of accumulated rewards completes the final piece of the lock-free update puzzle, preventing local locks from becoming a system performance bottleneck.

[0021] The introduction of this feature enables the present invention to stably support high-concurrency search and maintain decision quality in large-scale state space search scenarios of wargaming simulations.

[0022] In some embodiments, the node selection strategy employs a modified UCB1 formula, wherein the node access count is calculated using real-time values ​​adjusted for virtual loss.

[0023] in, Real-time access counts including virtual loss. To accumulate rewards, To explore constants, This is the visit count for the parent node.

[0024] The above technical solution specifies the node selection strategy used in the selection phase, namely the improved UCB1 formula. The core change in this formula is that it replaces the node access count with a "real-time access count including virtual loss." It is used in the calculation while preserving the exploration constant. Parent node visit count and cumulative rewards It incorporates the basic elements of the traditional UCB1 formula. Within the parallel Monte Carlo tree search framework defined by this feature, it plays a crucial role in balancing exploration and utilization. Its computational results determine the path selection of threads within the search tree, thereby influencing the convergence direction and diversity of the overall search behavior.

[0025] The essential feature of this technology is that it directly incorporates the real-time access count changes generated by the virtual loss mechanism into the utility evaluation function of node selection, enabling the UCB1 formula to dynamically respond to the instantaneous access heat fluctuations during the concurrent search process, thereby achieving real-time coordination between the node selection strategy and the load balancing guided by virtual loss.

[0026] First, the node visit count in the traditional UCB1 formula This reflects the cumulative historical access frequency of a node since its creation, and its update depends on the statistical accumulation during the backtracking phase after a complete iteration. In the parallel search framework of this invention, the virtual loss mechanism immediately deducts the access count through an atomic operation when a thread accesses a node. This generates a real-time access count that includes virtual loss. This feature will By directly substituting into the UCB1 formula, the instantaneous popularity of a node (i.e., the temporary access frequency adjusted for virtual loss) can immediately influence subsequent node selection decisions. Unlike traditional MCTS where node selection is based solely on historical cumulative statistics and has a lag, this improvement achieves real-time linkage between the selection strategy and the concurrent load status.

[0027] Secondly, this improvement forms a functional closed-loop synergy with the virtual loss mechanism. The virtual loss mechanism temporarily reduces the attractiveness of a node to the current thread through atomic deduction, its purpose being to guide concurrent threads to other branches; and the UCB1 formula in this feature is based on Performing calculations ensures that the effect of this reduced attractiveness is immediately perceived and responded to by other threads when selecting nodes. Specifically, when a thread applies a virtual loss to a node, the node's... The instantaneous decrease leads to the exploration term in the UCB1 formula. Increase, while utilizing the item This may also change, thereby comprehensively reducing the probability that the node will be selected by other threads. This real-time feedback mechanism enables the virtual loss diversion effect to be realized immediately, avoiding the correction delay problem caused by multiple threads still accessing the same node due to the lag in the selection strategy.

[0028] Furthermore, in this feature Its real-time performance is compatible with the underlying implementation of atomic operations. Because... It is stored using atomic integers and dynamically maintained by atomic subtraction / addition operations. Reading it from the UCB1 formula requires no locking, allowing different threads to concurrently calculate the scores of different nodes, and each thread reads... The values ​​are all instantaneous snapshots at a certain moment. This lock-free concurrent reading method not only ensures high throughput in the node selection process, but also avoids the additional synchronization overhead caused by strong consistency requirements by accepting instantaneous values ​​probabilistically, which is in line with the inherent fault-tolerant characteristics of Monte Carlo tree search as an approximate decision-making algorithm.

[0029] In summary, the essence of this technical feature is to transform the instantaneous access fluctuations introduced by the virtual loss mechanism into a real-time basis for node selection through an improved UCB1 formula. Its beneficial effects are: the node selection strategy can dynamically respond to load changes during concurrent search, allowing the traffic diversion and guidance effect of virtual loss to take effect immediately; the balance between exploration and utilization is no longer based on static judgments of lag statistics, but rather an adaptive process coupled with the real-time access behavior of threads; simultaneously, this improvement maintains good compatibility with the underlying mechanism of lock-free atomic operations without introducing additional synchronization overhead.

[0030] This technical feature will be achieved by... By introducing the UCB1 formula, this invention enables the selection phase to respond in real time to changes in access popularity caused by virtual losses, thereby unifying concurrency control methods and path optimization decisions within the same computational framework.

[0031] In some embodiments, the work-stealing thread pool uses a work-stealing algorithm to dynamically balance the load of each thread, encapsulating each legal action under the root node as an independent parallel search task, which is dynamically scheduled by the thread pool to be executed by an idle thread.

[0032] The above technical solution defines the task distribution and execution mechanism of S2, clarifies the construction method of parallel search tasks (encapsulating each legal action under the root node into an independent parallel search task) and the thread pool scheduling strategy (using a work-stealing algorithm to dynamically balance the load of each thread, and dynamically scheduling idle threads to execute the task). This feature is responsible for transforming the branch search requirements at the root node of the game tree into independent computing units that can be executed in parallel, and realizing the effective utilization of computing resources in a multi-core processor environment through the thread pool mechanism.

[0033] The essential feature of this technology is that it refines the task granularity of parallel search to the action level, which corresponds one-to-one with the root node and its child nodes, and adopts the work-stealing algorithm as the core mechanism for thread scheduling, thereby constructing a parallel computing framework that can adapt to the imbalance of the branch structure of the game tree.

[0034] First, this feature encapsulates each legitimate action under the root node as an independent parallel search task. This task partitioning method differs from implementation paths that treat the entire Monte Carlo tree search as a single task and only parallelize internal iterations, and also from static partitioning methods that split tasks based on a fixed number of nodes. Its core design lies in using the child nodes of the root node as natural partitioning boundaries, with each legitimate action corresponding to an independent search subtree. The search processes between subtrees are statistically independent (results are only aggregated at the root node). Since the number of legitimate actions in wargaming scenarios is typically in the tens to hundreds, and the depth and branch density of the subtrees corresponding to each action may vary significantly, this action-level task granularity allows parallel scheduling to allocate load in relatively fine units, avoiding the scheduling inflexibility problem caused by overly coarse task granularity.

[0035] Secondly, this feature employs a work-stealing algorithm to dynamically balance the load across threads. The core mechanism of the work-stealing algorithm is that each thread maintains a double-ended queue to store the tasks assigned to it. Threads preferentially retrieve tasks from the head of the queue for execution. When a thread's queue is empty, it does not idle and wait, but instead "steals" tasks from the tail of other threads' queues. This mechanism differs from traditional static task allocation models (such as pre-allocating tasks evenly to each thread) or models where a central scheduler uniformly distributes tasks. In wargame scenarios, the subtree search difficulty corresponding to different legal actions may vary significantly: some actions may quickly lead to the endgame, while others may generate complex game branches. If static allocation is used, some threads may be busy for a long time due to handling complex branches, while other threads become idle early due to handling simple branches, resulting in wasted computing power. The work-stealing algorithm achieves dynamic load rebalancing by allowing idle threads to actively steal tasks from busy threads. Essentially, it shifts the responsibility of load balancing from static pre-allocation to runtime adaptive adjustment.

[0036] Furthermore, the description of "dynamically scheduled to idle threads by the thread pool" in this feature works synergistically with the work-stealing algorithm. The thread pool, as the execution container for tasks, manages the lifecycle of its internal threads uniformly, avoiding the overhead of frequent thread creation and destruction. The work-stealing algorithm, as the scheduling strategy, determines how tasks flow among threads within the pool. Together, these two mechanisms enable the system to achieve distributed load balancing without a central scheduler. Unlike scheduling methods based on shared queue locking, the work-stealing algorithm's stealing operation only involves local locking at the tail of the target thread queue, with a stealing frequency far lower than the task execution frequency, thus keeping synchronization overhead at a low level.

[0037] In summary, the essence of this technology is to construct a parallel computing framework that adapts to the imbalance of branch structures by combining action-level task decomposition with work-stealing scheduling. Its beneficial effects are as follows: the task granularity naturally matches the branch structure of the game tree root node, avoiding the loss of scheduling flexibility caused by improper task partitioning; the work-stealing algorithm enables dynamic load balancing among threads, effectively avoiding idle computing power due to differences in search branch difficulty; the thread pool mechanism uniformly manages thread lifecycles, reducing system resource overhead; this scheduling framework provides a parallel execution environment for subsequent virtual loss-guided selection and lock-free backtracking, allowing each search task to run independently, with results only aggregated at the root node, avoiding mutual interference between tasks.

[0038] The introduction of this technical feature enables the present invention to fully utilize the computing resources of multi-core CPUs in large-scale state space search scenarios of wargame simulations, while maintaining the ability to adapt to the imbalance of the game tree branch structure.

[0039] In some embodiments, the method further includes a state object reuse and differential cloning mechanism, applied to the selection phase and the simulation phase: A pre-built pool of state objects is used to borrow state instances at the start of each Monte Carlo tree search iteration. During node traversal and action application in the selection phase, and during deduction stepping in the simulation phase, a new copy of the state is generated using differential copying technology. Only the changed fields are copied, while the unchanged fields share references. At the end of each iteration, the state instance is reset and returned to the object pool.

[0040] The aforementioned technical solution defines a state management mechanism and clarifies the implementation methods for state object reuse and differential cloning: a state object pool is pre-built, and state instances are borrowed from the pool at the start of each Monte Carlo tree search iteration; during node traversal and action application in the selection phase, and during deduction stepping in the simulation phase, a new state copy is generated using differential copying technology, copying only the changed fields, while unchanged fields share references; at the end of each iteration, the state instance is reset and returned to the object pool. This feature addresses the memory allocation overhead and garbage collection pressure issues caused by frequent object creation and destruction in high-frequency state cloning scenarios.

[0041] The essential feature of this technology lies in combining the object pool reuse mechanism with differential cloning technology, and limiting its application to the two stages with the most intensive state transitions: the selection stage and the simulation stage. This results in the construction of an efficient state management scheme for the Monte Carlo tree search iterative process.

[0042] First, this feature pre-builds a pool of state objects and borrows state instances from the pool at the start of each Monte Carlo tree search iteration, resetting and returning them at the end of the iteration. This design differs from traditional implementations that create new objects using the `new` operator during each state transition and rely on garbage collection for automatic reclamation. Its core lies in aligning the object lifecycle with the search iteration cycle: state instances are borrowed at the start of an iteration, reused repeatedly throughout the iteration (by generating state copies through differential cloning), and returned for reuse at the end of the iteration. Since Monte Carlo tree searches typically require thousands or even tens of thousands of iterations, creating new objects in each iteration would result in a large number of temporary objects residing in the heap, triggering frequent garbage collection. This feature reduces the number of object creations from the order of magnitude of iterations to the order of magnitude of the object pool capacity through the object pool, thereby reducing the garbage collection frequency from once every thousand iterations to once every ten thousand iterations. The resulting technical effect is a significant reduction in pause time in high-frequency search scenarios and an improvement in search throughput.

[0043] Secondly, this feature employs differential copying technology to generate new state copies, copying only the changed fields while keeping unchanged fields referenced. This technique differs from traditional full cloning (deep copying all fields) or state copying methods that rely on serialization and deserialization. In wargaming scenarios, battlefield states typically contain a large number of fields (such as map grids, multiple attributes of dozens of combat units), while a single action application often modifies only a few fields (such as the coordinates and health of a single combat unit). Full cloning means copying the entire state object with each state transition, resulting in a large number of redundant memory operations; while differential cloning records the set of changed fields, copying only the changed fields, while unchanged fields directly reference the corresponding data from the previous state. This copy-on-write approach reduces the memory copy size of a single state clone from O (state size) to O (number of changed fields). In typical wargaming scenarios, the number of changed fields is usually only 5%-10% of the total number of fields. The resulting technical effects are a significant reduction in state transition overhead during the selection and simulation phases, an increase in the number of simulation steps per unit time, and an expansion of search depth and breadth.

[0044] Furthermore, this feature explicitly applies state object reuse and differential cloning mechanisms to the selection and simulation phases. The selection phase involves multiple action applications from the root node to the leaf node, each requiring the generation of a new state copy. The simulation phase involves continuous deduction from the current node to the final state or maximum depth, also requiring multiple state transitions. These two phases are the most state-cloning-intensive stages in the Monte Carlo tree search iteration, with the number of state transitions proportional to the search depth (typically dozens of steps). This feature forms a functional division with the backtracking phase: the selection and simulation phases focus on the forward evolution of states, requiring high-frequency state cloning support; the backtracking phase focuses on the backward propagation of statistical information and does not involve state cloning. This division of labor allows for concentrated optimization resources, avoiding unnecessary complexity.

[0045] Finally, the "reset state instance and return it to the object pool" operation in this feature ensures the sustainable reuse of the object pool. The reset operation restores the fields of the state instance to their initial state or clears the change records, allowing the instance to be safely borrowed by subsequent iterations; the return operation puts the instance back into the pool for use by other threads or subsequent iterations. This closed-loop management is compatible with the concurrent model of "each parallel search task independently executing multiple iterations": because the object pool is implemented using a thread-safe blocking queue (such as ArrayBlockingQueue), different threads can borrow different instances from the pool without interfering with each other; the return at the end of the iteration ensures the cyclical reuse of instances.

[0046] In summary, the essence of this technology is to construct an efficient state management mechanism for the Monte Carlo tree search iterative process by combining object pool reuse and differential cloning. Its beneficial effects are as follows: object pool reuse reduces the number of object creations from the order of magnitude of iterations to the order of magnitude of the object pool capacity, significantly reducing garbage collection frequency; differential cloning reduces the amount of memory copying for a single state transition from a full copy to a modified copy, significantly improving state transition efficiency; this mechanism is applied to the selection and simulation phases, achieving a match between optimized resources and performance bottlenecks; the reuse closed loop is compatible with parallel search models, ensuring stable operation in high-concurrency scenarios.

[0047] This technology, along with parallel task splitting, virtual loss mechanism, and lock-free backtracking, forms a complete performance optimization system: task splitting and work stealing solve the problem of multi-core computing power utilization; virtual loss and lock-free backtracking solve the problems of concurrency control and statistical updates; and state object reuse and differential cloning solve the problems of memory allocation and copying overhead. The synergistic optimization of these three aspects across different dimensions enables this invention to stably support high-concurrency search iterations in large-scale state space scenarios of wargaming simulations with low garbage collection pauses and high state transition efficiency. This is a key technical support for achieving the dual objectives of "avoiding garbage collection pauses caused by high-frequency object creation" and "improving the effectiveness of simulated trajectories" in the invention's purpose.

[0048] In some embodiments, the method further includes a timeout circuit breaker mechanism: when the total time taken for a single decision exceeds a preset threshold, all unfinished parallel search tasks are terminated, and the action corresponding to the child node with the highest number of visits under the root node is output based on the statistical results of the currently completed tasks.

[0049] The above technical solution defines the decision termination and output mechanism and clarifies the implementation method of timeout circuit breaking: when the total time of a single decision exceeds a preset threshold, all unfinished parallel search tasks are terminated, and the action corresponding to the child node with the highest number of visits under the root node is output based on the statistical results of the currently completed tasks. This feature is responsible for establishing a balance between real-time constraints and search quality, ensuring that the system can still output decision instructions on time even in extremely complex scenarios.

[0050] The essential feature of this technology lies in its deep coupling with the task interruption capability in the parallel search architecture, achieving a configurable balance between search depth and real-time response.

[0051] First, this feature introduces a time threshold as the primary condition for decision termination, unlike traditional Monte Carlo tree search which relies solely on a fixed number of iterations or statistical convergence criteria. In wargaming scenarios, the real-time requirement for decision-making is a rigid constraint—the simulation system must output instructions within the round time window; otherwise, the entire simulation process will be interrupted. Traditional methods, if preset with a fixed number of iterations, cannot adapt to fluctuations in search complexity under different situations: simple situations may prematurely meet the iteration count, wasting computational resources, while complex situations may output low-quality decisions due to insufficient iterations. This feature, by using a time threshold as the primary termination condition, aligns the search time with the round time window, ensuring that the system can output decisions within the time limit under any circumstances.

[0052] Secondly, the operation of "terminating all unfinished parallel search tasks" when a timeout occurs is deeply coupled with the parallel search task structure defined in S2 and the execution model of the work-stealing thread pool in S3. Since each legitimate action is encapsulated as an independent parallel search task, and the tasks are statistically independent (results are only aggregated at the root node), the timeout circuit breaker mechanism can broadcast a termination signal to all tasks through the thread pool's interrupt interface (such as shutdownNow()). Upon detecting an interruption, each task immediately stops its current iteration and returns the completed partial statistical results. This design differs from treating the entire Monte Carlo tree search as a single, uninterruptible task. Its advantage lies in the task-level interruption granularity, which allows completed search results (statistical information of a portion of the subtree) to be preserved and utilized, rather than being completely discarded. The resulting technical effect is that even if a timeout occurs, the system can still output the current optimal solution based on the completed partial iterations of each task, avoiding decision gaps or regression to the default strategy due to timeouts.

[0053] Furthermore, after a timeout, this feature "outputs the action corresponding to the child node with the highest number of visits under the root node based on the currently completed statistical results," a rule consistent with the output rule for normal termination. Its core design ensures consistency in decision-making logic—whether the search terminates normally due to reaching the required number of iterations or is forcibly terminated due to a timeout circuit breaker, the output criterion is always based on the number of visits to the root node's child nodes. This uniformity allows the timeout circuit breaker mechanism to be viewed as an early truncation of the search process under time constraints, rather than a heterogeneous degradation strategy. Since the convergence characteristic of Monte Carlo tree search is that nodes with higher visit counts have a higher probability of becoming the optimal solution, even if the search does not fully converge, the result selected based on the current number of visits still has statistical validity. The resulting technical effect is that the timeout circuit breaker mechanism maintains a positive correlation between decision quality and search progress while ensuring real-time performance, avoiding a precipitous drop in decision quality due to timeouts.

[0054] Finally, the configurability of the "preset threshold" endows the system with the ability to adapt to different real-time requirements. Wargaming scenarios may include turn time requirements of different granularities (e.g., rapid simulation mode requires response times in the hundreds of milliseconds, while detailed simulation mode allows for calculations in the seconds). By adjusting the time threshold, this invention can flexibly balance search depth and response speed in different deployment environments. This configurability works synergistically with mechanisms such as parallel task splitting and work-stealing scheduling: the time threshold determines the total search budget, work-stealing scheduling is responsible for maximizing computing power utilization within the budget, and virtual loss and lock-free backtracking are responsible for ensuring search efficiency in a multi-threaded environment, together forming an optimization system oriented towards real-time constraints.

[0055] In summary, the essence of this technical feature is to use a time threshold as a priority condition for decision termination, and combine it with the task interruption capability and partial result utilization mechanism of the parallel search architecture to construct a circuit breaker mechanism that ensures real-time responsiveness. Its beneficial effects are as follows: it can guarantee the output of decision instructions within the round time window under any complex situation, meeting the rigid real-time requirements of the wargaming system; the task-level interruption and partial result utilization mechanism preserves completed search results, avoiding a sharp drop in decision quality due to timeouts; the consistency of output rules ensures the predictability of decision behavior; and the configurable threshold gives the system the ability to adapt to different real-time requirements.

[0056] This feature enables the present invention to prioritize real-time constraints while pursuing search depth and decision quality, ensuring that the system can operate stably in the highly real-time application scenario of wargaming.

[0057] In some embodiments, the simulation phase employs a heuristic random strategy, the heuristic rules of which include prioritizing attacks on high-value targets within range and prioritizing the capture of strategic points.

[0058] The aforementioned technical solution defines the heuristic stochastic strategy employed in the simulation phase and clarifies the specific content of the heuristic rules, including prioritizing attacks on high-value targets within range and prioritizing the capture of strategic points. This feature plays a crucial role in ensuring the quality of reward value generation during the Monte Carlo tree search iterative process. By introducing domain knowledge to guide the stochastic deduction process, it makes the simulation trajectory more closely resemble real game behavior, thereby improving the accuracy of reward value evaluation and simulation efficiency.

[0059] The essential feature of this technology lies in embedding prior knowledge from the field of wargaming into the simulation stage of Monte Carlo tree search in the form of heuristic rules. While maintaining the exploratory nature of stochastic strategies, it introduces guided decision preferences, thereby achieving a balance between simulation quality and computational efficiency.

[0060] First, this feature employs a heuristic stochastic strategy in the simulation phase, unlike the purely stochastic strategy (uniformly randomized selection among all legal actions) used in traditional Monte Carlo tree search. While purely stochastic strategies offer theoretical advantages such as simplicity and unbiased estimation, in scenarios like wargaming with high branching factors and long game sequences, the simulated trajectories often deviate significantly from real-world combat behavior, leading to excessively high variance in reward value assessments and requiring extensive simulations to converge to reliable results. This feature introduces two heuristic rules: "prioritizing attacks on high-value targets within range" and "prioritizing the capture of strategic points," which bias the action choices during the simulation towards a more tactically rational direction. The destruction of high-value targets (such as command vehicles and main battle tanks) often has a decisive impact on the course of the battle; prioritizing attacks on such targets makes the simulated trajectory more consistent with priority judgments in real combat. The capture of strategic points (such as high ground and supply points) often affects the effectiveness of subsequent operations; prioritizing the capture of such points allows the simulation to reflect the impact of terrain and resource factors on the game outcome. The resulting technical effect is that the reward value generated in a single simulation is closer to the actual game result, the effectiveness of the simulation trajectory is improved, thus obtaining a more stable statistical estimate under the same number of simulations, or reducing the number of simulations required to reach the same statistical confidence level.

[0061] Secondly, the "heuristic stochastic strategy" emphasizes the combination of heuristic rules and randomness, rather than being driven by completely deterministic rules. This design differs from purely rule-based decision-making systems (such as expert systems)—the latter rely entirely on deterministic logic and lack adaptability to unknown situations. In this feature, heuristic rules only serve as probabilistic bias factors: when there are high-value targets within range, prioritizing attacks is not mandatory, but rather increases the weight of attack actions; when there are strategic points that can be captured, prioritizing capture is also a probabilistic bias rather than a mandatory choice. The preservation of randomness allows the simulation process to still explore action spaces not pointed to by heuristic rules, avoiding the simulation trajectory from becoming too rigid due to overly rigid rules, thus maintaining the Monte Carlo tree search's ability to explore unknown strategies. The resulting technical effect is that while improving efficiency by utilizing domain knowledge in the simulation phase, the possibility of discovering unexpected strategies is still retained, echoing the balance of "exploration and utilization" in the strategy selection phase.

[0062] Furthermore, the heuristic rules in this feature are deeply aligned with the domain characteristics of wargaming scenarios. As a form of military simulation and confrontation, wargaming naturally incorporates tactical principles such as prioritizing attacks on high-value targets and controlling key terrain. This feature explicitly encodes this domain knowledge into action preferences during the simulation phase, enabling the simulation process to reflect general rationality at the tactical level. Compared to the difficulty in extracting universal heuristic rules in general game scenarios (such as Go and chess), the mature tactical principles existing in the wargaming domain provide a feasible foundation for designing heuristic strategies. The resulting technical effect is that this feature can fully utilize prior knowledge in the wargaming domain, transforming human tactical experience into an efficiency-enhancing factor for algorithms, achieving a beneficial combination of domain knowledge and statistical learning methods.

[0063] Finally, the application of heuristic rules in this feature synergizes with the parallel search architecture. Since the simulation phase is one of the most computationally expensive parts of the Monte Carlo tree search iteration (typically accounting for over 70% of the total computation), improvements in simulation efficiency directly translate into increased overall search throughput. Under the parallel search architecture, each thread executes the simulation independently. The reduction in single-simulation time and the decrease in reward variance brought about by the heuristic strategy further amplify the effect of parallel acceleration. Simultaneously, the heuristic rules and the virtual loss mechanism form a functional division: the virtual loss mechanism ensures search breadth through concurrency control, while the heuristic simulation strategy ensures search depth and convergence speed by improving simulation quality. Both work together to serve the core objective of improving the quality of decisions per unit time.

[0064] In summary, the essence of this technology is to embed domain knowledge into the simulation phase in the form of heuristic rules, guiding the simulation trajectory towards tactical rationality while retaining randomness. Its beneficial effects are: improved accuracy in reward value evaluation for a single simulation, enhanced effectiveness of the simulation trajectory, and increased stability of statistical estimation under the same number of simulations; the improved simulation efficiency indirectly reduces the number of iterations required to reach convergence, or allows the system to complete more effective simulations within a fixed time budget; the combination of heuristic rules and randomness maintains the ability to explore unknown strategies, avoiding the rigidity of deterministic rule systems.

[0065] This technical feature is an optimization method used in this invention to improve the simulation quality of Monte Carlo tree search. This feature, together with the virtual loss mechanism, lock-free backtracking, and work-stealing scheduling in the overall technical solution, forms a complete efficiency improvement system: work-stealing scheduling solves the problem of computing power utilization, the virtual loss mechanism solves the problem of concurrency distribution, lock-free backtracking solves the problem of statistical update overhead, and the heuristic simulation strategy solves the problem of single-simulation quality and efficiency. These four elements play roles in task scheduling, concurrency control, statistical updates, and simulation evaluation, respectively, jointly supporting an intelligent game decision-making method that can operate efficiently and converge quickly in large-scale state-space scenarios of wargames. The introduction of this feature allows this invention to effectively absorb and utilize prior knowledge in the field of wargames while maintaining the general exploration capabilities of Monte Carlo tree search.

[0066] According to another aspect of the present invention, a wargaming intelligent game decision-making system is provided, the system comprising, based on the above-described method: The situational awareness and state serialization module is used to acquire the current battlefield situation, construct the root node of the game tree, and enumerate all legal actions; The parallel search scheduling module is used to treat each legal action as a child node of the root node and create an independent parallel search task for each child node, which is then distributed for execution through a work-stealing thread pool. The lock-free game tree management module is used to execute various parallel search tasks. Each parallel search task independently performs multiple Monte Carlo tree search iterations, and each iteration includes: Selection phase: Starting from the root node, traverse downwards to the leaf node according to the node selection strategy. During the traversal, apply virtual loss deduction to the access count of the currently visited node through atomic operations to temporarily reduce the probability that the node will be selected by other threads. Expansion phase: If a leaf node is not fully expanded and is not the final stage, add a new child node to it; Simulation Phase: Starting from the current node, the simulation proceeds to the endgame or maximum depth using a heuristic random strategy to obtain reward values; Backtracking phase: From bottom to top along the search path, perform virtual loss cancellation, access count accumulation and reward value accumulation on each node on the path through atomic operations to achieve lock-free update; The decision-making and output module is used to summarize the statistical results of all parallel search tasks when the termination condition is met, and select the action corresponding to the child node with the largest number of visits under the root node as the final decision instruction.

[0067] In order to better utilize the above methods, this application proposes a wargaming intelligent game decision-making system. Each module corresponds to a step of the above methods, and its specific principles have been described above and will not be repeated here.

[0068] According to another aspect of the present invention, a wargaming intelligent game decision-making device is provided, comprising: At least one processor and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0069] In the above technical solution, to better operate and process the method, the method is stored in memory, and the processor executes the stored method. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here.

[0070] According to another aspect of the present invention, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described method.

[0071] In the above technical solution, to better operate and use the method, the method is stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0073] Figure 1 This is a flowchart illustrating an embodiment of the intelligent game decision-making method for wargaming simulation according to the present invention. Figure 2This is a schematic diagram of an embodiment of the intelligent game decision-making system for war game simulation according to the present invention. Detailed Implementation

[0074] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] Example 1 Please see Figure 1 A wargaming intelligent game decision-making method includes the following steps: S1. Obtain the current battlefield situation, construct the root node of the game tree, and enumerate all legal actions; For example, the wargaming engine pushes battlefield situation data in JSON format to the decision service each round. After receiving the data, the situation analysis module constructs an immutable GameState object. GameState contains map grid information (100×100 grid), combat unit attributes (position coordinates, health, ammunition, morale), and current round information.

[0076] To reduce memory allocation overhead, a pre-built state object pool (ArrayBlockingQueue with a capacity of 500) is constructed. Idle GameState instances are borrowed from the pool, and the root node Root is initialized through a deep copy, with the root node's access count recorded. Initialize to 0, cumulative rewards Initialize to 0.0.

[0077] The decision engine calls the wargaming rules interface to enumerate all legal actions for the Red side in the current situation. In this embodiment, a total of 56 legal actions are generated in a certain round, including instructions for movement, attack, capture, and resupply of each combat unit.

[0078] S2. Treat each legal action as a child node of the root node, and create an independent parallel search task for each child node, which is then distributed for execution through a work-stealing thread pool. In this embodiment, the work-stealing thread pool uses a work-stealing algorithm to dynamically balance the load of each thread, encapsulating each legal action under the root node into an independent parallel search task, which is dynamically scheduled by the thread pool to be executed by an idle thread.

[0079] For example, the system for each legitimate action Create a SearchTask object, which is responsible for... Perform a subtree search starting from the RecursiveTask. <integer>It encapsulates the initial state and search parameters corresponding to the action.

[0080] The main thread calls the `ForkJoinPool.invokeAll(tasks)` method to distribute 56 `SearchTask` tasks to the work-stealing thread pool. The thread pool has a default parallelism of 16 (matching the number of CPU cores) and uses a work-stealing algorithm to dynamically balance the load of each thread. When a thread completes its own task, it steals a task from the tail of other threads' task queues to ensure that all threads are always busy, maximizing CPU utilization.

[0081] After dispatching tasks, the main thread starts an independent timer to monitor the total time taken for a single decision. The timer will activate when it reaches a preset threshold. (In this embodiment, the timeout is set to 1000ms). At this time, the main thread calls the `ForkJoinPool.shutdownNow()` method to send an interrupt signal to all `SearchTask`s, terminating any unfinished search tasks. Based on the currently completed statistical results, it outputs the optimal action, implementing a timeout circuit breaker mechanism to ensure real-time response even under extremely complex conditions. Each `SearchTask` checks its interrupt status in its `compute()` method using `Thread.currentThread().isInterrupted()`. If an interruption is detected, it immediately terminates the iteration and returns the completed statistical results.

[0082] S3. Each parallel search task independently executes multiple Monte Carlo tree search iterations, each iteration including: Selection phase: Starting from the root node, traverse downwards to the leaf node according to the node selection strategy. During the traversal, apply virtual loss deduction to the access count of the currently visited node through atomic operations to temporarily reduce the probability that the node will be selected by other threads. Expansion phase: If a leaf node is not fully expanded and is not the final stage, add a new child node to it; Simulation Phase: Starting from the current node, the simulation proceeds to the endgame or maximum depth using a heuristic random strategy to obtain reward values; Backtracking phase: From bottom to top along the search path, perform virtual loss cancellation, access count accumulation and reward value accumulation on each node on the path through atomic operations to achieve lock-free update; In this embodiment, the node selection strategy adopts an improved UCB1 formula, wherein the node access count is calculated using real-time values ​​adjusted for virtual loss.

[0083] in, Real-time access counts including virtual loss. To accumulate rewards, To explore constants, This is the visit count for the parent node.

[0084] In this embodiment, the virtual loss deduction and offset are specifically as follows: the node access count is stored using atomic integers; when a thread enters a node, an atomic subtraction operation is performed to deduct a preset virtual loss value; when a thread leaves a node, an atomic addition operation is performed to add back the virtual loss value and additionally accumulate the real access count; wherein, the virtual loss value is a configurable positive integer, and its value range dynamically increases with the increase of the total number of concurrent threads.

[0085] In this embodiment, the method further includes a state object reuse and differential cloning mechanism, applied to the selection phase and the simulation phase: A pre-built pool of state objects is used to borrow state instances at the start of each Monte Carlo tree search iteration. During node traversal and action application in the selection phase, and during deduction stepping in the simulation phase, a new copy of the state is generated using differential copying technology. Only the changed fields are copied, while the unchanged fields share references. At the end of each iteration, the state instance is reset and returned to the object pool.

[0086] In this embodiment, the simulation phase employs a heuristic random strategy, and the heuristic rules include prioritizing attacks on high-value targets within range and prioritizing the capture of strategic points.

[0087] For example, each SearchTask independently executes multiple Monte Carlo tree search iterations, and the worker thread executes the compute() method, entering the MCTS loop (select -> expand -> simulate -> backtrack) until the preset number of iterations per task or time threshold is reached.

[0088] (1) Selection stage The thread starts from the root node and traverses downwards to the leaf nodes according to the node selection strategy. When selecting child nodes, this embodiment uses a modified UCB1 formula as the node selection strategy:

[0089] in, For real-time access counts that include virtual loss, To accumulate rewards, To explore constants, To prevent the minimum value of division by zero, This is the visit count for the parent node. The exploration constant can be set to... Alternatively, an adaptive algorithm can be used to dynamically adjust and balance exploration and exploitation.

[0090] Node statistics are stored using atomic variables: visitCount is of type AtomicInteger, and totalReward is of type AtomicDouble. Virtual loss value. Dynamically adjusts based on the current number of active threads in the thread pool: The system monitors the number of active threads in real time. ,when hour, Dynamically adjusted to 5; when hour, Adjust to 2; in other cases The value is kept at 3. The dynamic adjustment mechanism ensures that a larger virtual loss is applied in high-concurrency scenarios to enhance the thread splitting effect, while a smaller virtual loss is applied in low-concurrency scenarios to avoid overexploration that would lead to a decrease in convergence speed.

[0091] When the thread decides to access the node At that time, atomic operations are performed immediately. This operation reduces the temporary access count of the node. The UCB value drops instantly, thereby reducing the node's attractiveness to other concurrent threads, forcing other threads to explore other unvisited or low-access branches, thus achieving implicit load balancing.

[0092] During the selection process, each time a thread moves downwards, a new copy of the state is generated using differential cloning technology: only the changed fields (such as the coordinates after movement, the reduced health) are copied, while the unchanged fields share references. In this embodiment, a single state clone copies an average of only 12 changed fields, which is about 15 times more efficient than a full clone (approximately 200 fields).

[0093] (2) Expansion phase When a thread reaches a leaf node, it checks if the node is a final node (e.g., one side is wiped out or the maximum number of rounds has been reached). If it is not a final node and the node has not been fully expanded, it selects an unvisited action from the current node's legal actions, creates a new child node (TreeNode), and initializes its visitCount and totalReward. The node's expansion state is marked with the volatile variable fullyExpanded to ensure visibility between multiple threads. The list of child nodes is stored using CopyOnWriteArrayList to ensure thread safety during concurrent additions.

[0094] (3) Simulation phase Starting from the current node, the thread uses a heuristic randomization strategy to quickly deduce the game to its end or reach its maximum depth. (This embodiment is set to 50 steps). During the simulation, each action selection prioritizes heuristic rules: when there is a high-value enemy target within range (such as a command vehicle or main battle tank), the attack command is executed first; when the morale of friendly units is below the threshold, the supply or retreat command is executed first; otherwise, the action is randomly selected from the remaining legal actions.

[0095] After the simulation is completed, the reward value is calculated based on the final outcome. A victory is worth 1.0, a defeat 0.0, and a draw 0.5. If the simulation reaches its maximum depth before ending, the situation scoring function (considering factors such as troop strength and resource dominance) is called to calculate the reward value. The simulation depth can be limited by setting a maximum number of simulation steps (e.g., 50 steps) or a time slice (e.g., 10ms) to prevent long-sequence simulations from blocking the thread.

[0096] (4) Retrospective phase Threads receive reward values Then, backtrack from bottom to top along the search path. For each node on the path... Perform atomic operations:

[0097] in, For the previously deducted virtual loss value, addAndGet(L + 1) performs two tasks simultaneously: adding back the previously deducted virtual loss value. This step also increments the actual visit count for this iteration. This step ensures that the final statistics for a node reflect the true search results while eliminating the interference of virtual loss.

[0098] This backtracking process employs a lock-free update method, allowing different threads to simultaneously perform atomic addition operations on different nodes without needing to lock the nodes. This completely eliminates the thread blocking caused by `synchronized` (a Java mutex synchronization primitive implemented based on the JVM's built-in monitor lock, which forces threads to acquire ownership of the object monitor before entering a synchronized block, thus ensuring atomicity and visibility in memory semantics. Its underlying implementation relies on lock flags in the object header, supporting a lock escalation mechanism from biased locking and lightweight locking to heavyweight locking to ensure thread safety when accessing shared resources in a multi-threaded concurrent environment). In actual testing, with 16 threads executing concurrently, the lock-free backtracking method improved throughput by approximately 3.2 times compared to the locking method.

[0099] After each iteration, the thread resets the borrowed GameState instance (restoring all fields to their initial values) and returns it to the object pool for reuse in subsequent iterations. The object pool works in conjunction with differential cloning: when returning the state, the GameState's reset() method iterates through the changedFields list, restores changed fields to their default values, clears the changedFields list, and then returns the instance to the ArrayBlockingQueue. This collaborative mechanism avoids performing new GameState() operations on hot search paths. The memory allocation for state cloning is reduced by approximately 85% in a single MCTS iteration, and the garbage collection frequency is reduced from once every 1000 iterations to once every 10000 iterations, significantly reducing the impact of GC pauses on search performance. (GC, Garbage Collection, is a mechanism by which the JVM automatically manages heap memory based on reachability analysis algorithms. It aims to reclaim unused objects to release resources. However, high-frequency object creation will accelerate the exhaustion of the Eden space in the young generation, thereby frequently triggering Minor GC and causing Stop-The-World (global pause), ultimately reducing system throughput due to the accumulated pause time.)

[0100] S4. When the termination condition is met, summarize the statistical results of all parallel search tasks and select the action corresponding to the child node with the largest number of visits under the root node as the final decision instruction.

[0101] In this embodiment, the method also includes a timeout circuit breaker mechanism: when the total time for a single decision exceeds a preset threshold, all unfinished parallel search tasks are terminated, and the action corresponding to the child node with the largest number of visits under the root node is output based on the statistical results of the current completion.

[0102] For example, the main thread waits for all SearchTasks to complete (or be interrupted by a timeout circuit breaker) using ForkJoinPool.awaitTermination(). Termination conditions include: the total search time reaching a preset threshold (800ms in this example), the total number of iterations reaching the upper limit (10000 times in this example), or the timeout circuit breaker being triggered (1000ms). Once all tasks are completed or the termination condition is triggered, the direct child nodes of the root node are traversed, and the number of visits is selected. The action corresponding to the largest child node is used as the final decision instruction. In this embodiment, the final selected action is "The 3rd Armored Company moves to coordinates (45,78) and attacks the enemy's 2nd Reconnaissance Platoon". The decision service sends the instruction to the wargaming client via the WebSocket interface to complete the single-round decision.

[0103] The core concept of this invention lies in the deep integration of parallel computing and lock-free concurrency technology to construct a Monte Carlo tree search method that can efficiently utilize the computing power of multi-core CPUs and achieve high real-time intelligent decision-making in ultra-large state space wargame scenarios. The innovation of this invention lies in fundamentally solving the problems of severe thread contention, single search path, and high memory overhead in traditional parallel MCTS under high concurrency environments through two mechanisms: "virtual loss-guided load balancing" and "lock-free game tree management based on atomic variables," thus achieving a balance between search efficiency, breadth, and depth.

[0104] This invention primarily addresses three technical problems in existing intelligent decision-making technologies for wargaming simulations: (1) Limitations of traditional rule systems: Static logic that relies on expert rule bases (such as the Drools engine) lacks adaptability, cannot effectively cope with dynamic changes and uncertain factors on the battlefield, and is difficult to generate intelligent game behavior with strong adversarial characteristics.

[0105] (2) Performance bottleneck of single-threaded MCTS: The state space complexity of war game simulation is far greater than that of Go. In a single-threaded environment, the number of simulations per unit time is extremely low, resulting in slow algorithm convergence speed, which cannot meet the requirements of war game simulation for real-time decision-making at the second or millisecond level.

[0106] (3) Conflicts and efficiency issues of multi-threaded concurrency: Directly using multi-threaded parallel search of shared game tree will cause serious lock contention, resulting in a decrease in system throughput; if not controlled, multiple threads will focus on searching the same high-value path, causing loss of search diversity and easily getting trapped in local optima.

[0107] To address the above problems, this invention proposes a technical solution, mainly including the following means: (1) Concurrency selection strategy based on Virtual Loss: When a thread accesses a node, an atomic operation immediately subtracts a preset "virtual loss" value (e.g., 3) from the node's access count. This is equivalent to instantly reducing the node's "attractiveness" to other threads (UCB value decreases), forcing other threads to automatically turn to explore other unvisited or less frequently visited branches. This is a clever implicit load balancing mechanism that effectively avoids thread clustering and ensures search diversity without the need for a global lock.

[0108] (2) Lock-free game tree management based on atomic variables: The traditional synchronized (synchronized is a mutual exclusion synchronization primitive implemented in Java based on the JVM's built-in monitor lock. It forces threads to acquire ownership of the object monitor before entering a synchronized block, thus ensuring atomicity and visibility in memory semantics. Its underlying mechanism relies on the lock flag in the object header, supporting a lock escalation mechanism from biased locking, lightweight locking to heavyweight locking to ensure thread safety when accessing shared resources in a multi-threaded concurrent environment) lock mechanism. Instead, it utilizes AtomicInteger and AtomicDouble from Java's java.util.concurrent.atomic package to store the core statistical information of nodes (access count N and cumulative reward Q). During backtracking updates, lock-free updates are performed using CAS (Compare-And-Swap) instructions, achieving thread safety while completely eliminating thread blocking and context switching overhead caused by lock contention.

[0109] (3) Parallel search scheduling and task splitting: Using Java's Fork / Join framework and work-stealing algorithm, each legal action under the root node is mapped to an independent RecursiveTask subtask. This mechanism can automatically distribute tasks evenly across multi-core CPUs for execution, maximizing CPU computing power utilization.

[0110] (4) State snapshot and memory optimization mechanism: Differential cloning (Diff Cloning) copies only the changed parts of the state object (such as coordinates and health) during the simulation process, while the unchanged parts share references, reducing memory copying overhead. Object pooling technology pre-allocates and reuses GameState objects, avoiding frequent creation and destruction of objects during high-frequency search, effectively reducing system pauses caused by garbage collection (GC). (GC, Garbage Collection, is a mechanism by which the JVM automatically manages heap memory based on reachability analysis algorithms, aiming to reclaim unused objects to release resources; however, high-frequency object creation will accelerate the exhaustion of the Eden space in the young generation, thereby frequently triggering Minor GC and causing Stop-The-World (global pause), ultimately reducing system throughput due to accumulated pause time).

[0111] Compared with the prior art, the present invention has the following beneficial effects: (1) Significantly improves parallel search efficiency and meets real-time decision-making requirements. This invention encapsulates each legal action under the root node into an independent parallel search task through a work-stealing thread pool, which is dynamically scheduled to be executed by idle threads. This fully utilizes the computing power of multi-core CPUs and significantly increases the number of Monte Carlo tree search iterations per unit time. Compared with the traditional single-threaded MCTS, this invention can complete intelligent decision-making in a large-scale state space in a shorter time, meeting the stringent real-time requirements of wargame simulations.

[0112] (2) Eliminating thread contention and ensuring search stability under high concurrency. This invention uses atomic integers and atomic double-precision floating-point types to store the statistical information of game tree nodes. It achieves lock-free updates of nodes through atomic operations, completely eliminating the thread blocking and context switching overhead caused by the use of synchronized lock mechanism in traditional multi-threaded MCTS (synchronized is a mutual exclusion synchronization primitive implemented by Java based on the JVM's built-in monitor lock. It constructs memory semantic guarantees of atomicity and visibility by forcing threads to acquire ownership of the object monitor before entering the synchronized block. Its underlying implementation relies on the lock flag in the object header and supports a lock expansion mechanism from biased locks, lightweight locks to heavyweight locks to ensure thread safety for accessing shared resources in a multi-threaded concurrent environment). Especially in the backtracking phase, lock-free updates are achieved through atomic operations, allowing multiple threads to update different nodes in parallel without interfering with each other, significantly improving the system throughput in high-concurrency scenarios.

[0113] (3) Implicit load balancing is achieved through a virtual loss mechanism, enhancing search diversity. This invention introduces a virtual loss mechanism during the selection phase of parallel search. When a thread accesses a node, its access count is temporarily deducted through atomic operations, dynamically reducing the node's attractiveness to other threads and forcing concurrent threads to automatically branch to unexplored or low-access-count branches. This mechanism achieves implicit load balancing between threads without the need for global locks or complex task scheduling, effectively avoiding the loss of search diversity and local optima problems caused by multiple threads concentrating on searching the same high-value path.

[0114] (4) Improved UCB1 formula balances exploration and utilization, enhancing decision quality. This invention introduces real-time access counts, including virtual loss, into the UCB1 formula, enabling the node selection strategy to dynamically respond to the impact of virtual loss and achieve a better balance between exploration and utilization. This improvement ensures that during parallel search, each thread can fully utilize existing high-value paths while continuously exploring potentially better branches, thereby improving the game-theoretic win rate of the final decision.

[0115] (5) State object reuse and differential cloning significantly reduce memory overhead. This invention reuses state instances in each Monte Carlo tree search iteration by pre-building a state object pool, avoiding garbage collection pauses caused by high-frequency object creation and destruction. At the same time, differential cloning technology is used to copy only the changed fields during state transition, while unchanged fields share references, which significantly reduces memory copying overhead and further improves search efficiency.

[0116] (6) Timeout Circuit Breaker Mechanism Ensures Real-Time System Response. This invention introduces a timeout circuit breaker mechanism during the decision generation process. When the time taken for a single decision exceeds a preset threshold, the unfinished search task is forcibly terminated, and the current optimal solution is output based on existing statistical results. This mechanism ensures that the wargaming system can still output decision commands on time even in extremely complex scenarios, meeting the real-time response requirements in military simulation and confrontation.

[0117] In summary, this invention effectively solves the technical problems of slow response speed, severe concurrency conflicts, insufficient search diversity, and high memory consumption in traditional wargame intelligent decision-making methods by comprehensively applying parallel task scheduling, virtual loss-guided load balancing, lock-free concurrent updates, state object reuse and differential cloning, and timeout circuit breaker mechanisms. It achieves efficient, stable, and real-time intelligent decision-making in large-scale state space game scenarios.

[0118] Example 2 Please see Figure 2 A wargaming intelligent game decision-making system, based on the above method, the system includes: The situational awareness and state serialization module is used to acquire the current battlefield situation, construct the root node of the game tree, and enumerate all legal actions; In this embodiment, this module is responsible for parsing the raw battlefield data (JSON / Protobuf format) transmitted from the wargaming engine. It also constructs immutable GameState objects, containing map grid information, combat unit attributes (position, health, ammunition, morale), resource status, and turn information. Furthermore, it integrates an object pool mechanism to pre-allocate GameState instances, avoiding frequent garbage collection (GC). (GC is a mechanism by which the JVM automatically manages heap memory based on reachability analysis algorithms, aiming to reclaim unused objects to release resources; however, high-frequency object creation accelerates the exhaustion of the Eden space in the young generation, thus frequently triggering Minor GC and causing Stop-The-World (STH) pauses, ultimately reducing system throughput due to accumulated pause time.)

[0119] The parallel search scheduling module is used to treat each legal action as a child node of the root node and create an independent parallel search task for each child node, which is then distributed for execution through a work-stealing thread pool. In this embodiment, the module constructs a work-stealing thread pool based on java.util.concurrent.ForkJoinPool. It is also responsible for mapping legitimate actions under the root node to independent RecursiveTask subtasks and dynamically balancing the load across threads.

[0120] The lock-free game tree management module is used to execute various parallel search tasks. Each parallel search task independently performs multiple Monte Carlo tree search iterations, and each iteration includes: Selection phase: Starting from the root node, traverse downwards to the leaf node according to the node selection strategy. During the traversal, apply virtual loss deduction to the access count of the currently visited node through atomic operations to temporarily reduce the probability that the node will be selected by other threads. Expansion phase: If a leaf node is not fully expanded and is not the final stage, add a new child node to it; Simulation Phase: Starting from the current node, the simulation proceeds to the endgame or maximum depth using a heuristic random strategy to obtain reward values; Backtracking phase: From bottom to top along the search path, perform virtual loss cancellation, access count accumulation and reward value accumulation on each node on the path through atomic operations to achieve lock-free update; In this embodiment, the module maintains a TreeNode structure in memory for the game. Node statistics (number of visits N, cumulative reward Q) are stored using java.util.concurrent.atomic.AtomicInteger and AtomicDouble, and lock-free updates are achieved using CAS instructions, completely eliminating thread blocking caused by synchronized (synchronized is a mutual exclusion synchronization primitive implemented in Java based on the JVM's built-in monitor lock. It forces threads to acquire ownership of the object monitor before entering a synchronized block, thus ensuring atomicity and visibility in memory semantics. Its underlying implementation relies on lock flags in the object header, supporting lock escalation mechanisms from biased locking, lightweight locking to heavyweight locking to ensure thread safety for accessing shared resources in a multi-threaded concurrent environment). The child node list uses CopyOnWriteArrayList to ensure thread safety during structural modifications. This module incorporates a lightweight rule-guided random strategy. Based on a non-completely random approach, heuristic rules (such as prioritizing attacks on high-value targets within range and prioritizing the capture of high ground) are added to improve the effectiveness of simulated trajectories. This module also supports fast state cloning, which copies only the difference data (Diff-based Cloning).

[0121] The decision-making and output module is used to summarize the statistical results of all parallel search tasks when the termination condition is met, and select the action corresponding to the child node with the largest number of visits under the root node as the final decision instruction.

[0122] In the local simulation, this module monitors the global search time and number of iterations. Once the termination condition is met, it selects the optimal action based on either the maximum number of visits (Robust Child) or the maximum average reward (Max Child) criterion. The decision instructions are then sent to the simulation client via a WebSocket interface.

[0123] In order to better utilize the above methods, this application proposes a wargaming intelligent game decision-making system. Each module corresponds to a step of the above methods, and its specific principles have been described above and will not be repeated here.

[0124] Example 3 A wargaming simulation intelligent game decision-making device includes: At least one processor and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method as described in one of the embodiments.

[0125] In the above technical solution, in order to better operate and process the method described in one of the embodiments, the method is stored in a memory, and the stored method is executed by a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated further here.

[0126] Example 4 A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in one of the embodiments.

[0127] In the above technical solution, to better operate and use the method described in one of the embodiments, the method is stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated further here.

[0128] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.< / integer>

Claims

1. A wargaming intelligent game decision-making method, characterized in that, Includes the following steps: S1. Obtain the current battlefield situation, construct the root node of the game tree, and enumerate all legal actions; S2. Treat each legal action as a child node of the root node, and create an independent parallel search task for each child node, which is then distributed for execution through a work-stealing thread pool. S3. Each parallel search task independently executes multiple Monte Carlo tree search iterations, each iteration including: Selection phase: Starting from the root node, traverse downwards to the leaf node according to the node selection strategy. During the traversal, apply virtual loss deduction to the access count of the currently visited node through atomic operations to temporarily reduce the probability that the node will be selected by other threads. Expansion phase: If a leaf node is not fully expanded and is not the final stage, add a new child node to it; Simulation Phase: Starting from the current node, the simulation proceeds to the endgame or maximum depth using a heuristic random strategy to obtain reward values; Backtracking phase: From bottom to top along the search path, perform virtual loss cancellation, access count accumulation and reward value accumulation on each node on the path through atomic operations to achieve lock-free update; S4. When the termination condition is met, summarize the statistical results of all parallel search tasks and select the action corresponding to the child node with the largest number of visits under the root node as the final decision instruction.

2. The intelligent game decision-making method for wargaming simulation as described in claim 1, characterized in that, The virtual loss deduction and offset are specifically as follows: the node access count is stored using atomic integers, and the cumulative reward is stored using atomic double-precision floating-point numbers; when a thread enters a node, an atomic subtraction operation is performed to deduct a preset virtual loss value; when a thread leaves a node, an atomic addition operation is performed to add back the virtual loss value and add an additional real access count; wherein, the virtual loss value is a configurable positive integer, and its value range dynamically increases with the increase of the total number of concurrent threads.

3. The intelligent game decision-making method for wargaming simulation as described in claim 1, characterized in that, The node selection strategy employs an improved UCB1 formula, where the node access count uses real-time values ​​adjusted for virtual loss in the calculation. in, Real-time access counts including virtual loss. To accumulate rewards, To explore constants, This is the visit count for the parent node.

4. The intelligent game decision-making method for wargaming simulation as described in claim 1, characterized in that, The work-stealing thread pool uses a work-stealing algorithm to dynamically balance the load of each thread, encapsulating each legal action under the root node as an independent parallel search task, which is dynamically scheduled by the thread pool to be executed by an idle thread.

5. The intelligent game decision-making method for wargaming simulation as described in claim 1, characterized in that, The method also includes a state object reuse and differential cloning mechanism, applied to the selection phase and the simulation phase: A pre-built pool of state objects is used to borrow state instances at the start of each Monte Carlo tree search iteration. During node traversal and action application in the selection phase, and during deduction stepping in the simulation phase, a new copy of the state is generated using differential copying technology. Only the changed fields are copied, while the unchanged fields share references. At the end of each iteration, the state instance is reset and returned to the object pool.

6. The intelligent game decision-making method for wargaming simulation as described in claim 1, characterized in that, The method also includes a timeout circuit breaker mechanism: when the total time for a single decision exceeds a preset threshold, all unfinished parallel search tasks are terminated, and the action corresponding to the child node with the largest number of visits under the root node is output based on the statistical results of the current completed tasks.

7. The intelligent game decision-making method for wargaming simulation as described in claim 1, characterized in that, The simulation phase employs a heuristic random strategy, which includes prioritizing attacks on high-value targets within range and prioritizing the capture of strategic points.

8. A wargaming intelligent game decision-making system, characterized in that, Based on the method according to any one of claims 1-6, the system comprises: The situational awareness and state serialization module is used to acquire the current battlefield situation, construct the root node of the game tree, and enumerate all legal actions; The parallel search scheduling module is used to treat each legal action as a child node of the root node and create an independent parallel search task for each child node, which is then distributed for execution through a work-stealing thread pool. The lock-free game tree management module is used to execute various parallel search tasks. Each parallel search task independently performs multiple Monte Carlo tree search iterations, and each iteration includes: Selection phase: Starting from the root node, traverse downwards to the leaf node according to the node selection strategy. During the traversal, apply virtual loss deduction to the access count of the currently visited node through atomic operations to temporarily reduce the probability that the node will be selected by other threads. Expansion phase: If a leaf node is not fully expanded and is not the final stage, add a new child node to it; Simulation Phase: Starting from the current node, the simulation proceeds to the endgame or maximum depth using a heuristic random strategy to obtain reward values; Backtracking phase: From bottom to top along the search path, perform virtual loss cancellation, access count accumulation and reward value accumulation on each node on the path through atomic operations to achieve lock-free update; The decision-making and output module is used to summarize the statistical results of all parallel search tasks when the termination condition is met, and select the action corresponding to the child node with the largest number of visits under the root node as the final decision instruction.

9. A wargaming simulation intelligent game decision-making device, characterized in that, include: At least one processor and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images