Multi-level game decision-making system fusing situation knowledge

Abstract

The invention provides a multi-level game decision-making system fusing situation knowledge, belongs to the field of game decision-making in artificial intelligence, and aims to solve the problem that multiple red-blue adversarial solution schemes are difficult to adapt to cross-domain multi-scene and complex multi-task migration multiplexing, generalization modeling of a game solution process is carried out based on a layering thought, and a multi-level game decision-making algorithm is obtained. Each layer of game problem solving comprises a game environment structured processor, a game information processor, a game state estimator, a game state deduction / simulator, a multi-level value transmission and strategy generator and a strategy action conversion executor, and different scenes can be quickly modeled; and a proper algorithm solving strategy is selected in each stage according to the general game solving step without repeatedly developing a new system, so that the aim of multi-scene multiplexing of one set of framework is fulfilled.

Description

Technical Field

[0001] The present invention belongs to the field of game decision-making in artificial intelligence, and specifically relates to a multi-level game decision-making system integrating situation knowledge. Background Art

[0002] As one of the important fields of artificial intelligence research, computer game decision-making inherits the original genes of artificial intelligence and tests the development level of artificial intelligence. As early as the 1940s, pioneers of computer science such as John von Neumann, the father of computer, Alan Turing, the father of artificial intelligence, and Shannon, the founder of information theory, began to explore machine games. Later, with the advent of IBM Deep Blue, which can play chess, and AlphaGO, which can play Go, the ability level of computer game programs has been catching up with and beginning to surpass human professional players. Game decision-making has demonstrated strong autonomous learning ability and development potential in confrontation problems in complex scenarios, gradually opening up the era of intelligence.

[0003] Whether in the research of the basic theory of game decision-making or the specific application of game problems, universality and generalization have always been one of the important goals of related work. However, the existing game algorithm solution is difficult to apply to the migration and reuse of cross-domain multi-scenario and complex multi-task, so it is urgent to study the common framework of games. By studying a modular and highly scalable common framework that integrates the existing algorithms for solving game problems, it is possible to quickly model the confrontation environment in different scenarios, and select the appropriate algorithm solution strategy at each stage according to the general game solution steps without repeatedly developing new systems, so as to achieve the purpose of multiple-scenario reuse of a set of frameworks, support the rapid expansion of complex multi-tasks, multi-scenarios, and multi-levels in the economy, politics, and military, and realize the rapid construction of game algorithms under differentiated scenarios and tasks. Summary of the invention

[0004] The present invention aims at the problem that existing game algorithm solutions are difficult to apply to multiple scenarios and complex multi-task migration and reuse. The present invention provides a multi-level game decision-making system that integrates situation knowledge.

[0005] The technical solution adopted by the present invention is:

[0006] A multi-level game decision system integrating situation knowledge, including multiple sub-game problem levels, each level including a game environment structured processor, a game information processor, a game state estimator and a game state deduction / simulator; and also including a multi-level value transfer and strategy generator and a strategy action conversion executor;

[0007] The game environment structured processor of each layer is used to abstract the game information into various structured data that can be recognized and processed by the computer, including confrontation state, unit action, object configuration and deduction strategy configuration; the confrontation state structured data is output to the game information processor, game state estimator and game state deduction / simulator, and the confrontation state includes the traversable area, obstacles, high-value points, unit point information and unit execution action; the unit action structured data is output to the game information processor, game state deduction / simulator and strategy action conversion executor, and the unit action is formed by the executable actions of each unit in the confrontation scenario, including attack and movement; the object configuration structured data is output to the game information processor, game state estimator, game state deduction / simulator and multi-level value transfer and strategy generator, and the object configuration includes the number, type and confrontation level of each unit; the deduction strategy configuration structured data is output to the game state estimator and game state deduction / simulator, and the deduction strategy configuration includes the game method type and method parameters;

[0008] The game information processor at each layer is used to understand and identify the object configuration structured data, build a static situation information system in the fog environment, and analyze and recognize the dynamic processing information of the enemy unit based on the confrontation state structured data and unit action structured data sorted out by the game environment structured processor, including future positions, behaviors, intentions and threats, and output the analysis and cognition results including static knowledge and dynamic processing information to the game state estimator;

[0009] The game state estimator of each layer scores the deduction results output by the game state deduction / simulator based on the confrontation state, object configuration and deduction strategy configuration structured data provided by the game environment structured processor and the analysis and cognition results provided by the game information processor, evaluates the quality of action execution, and outputs the current situation score result to the game state deduction / simulator; it is also used to re-score the deduction results output by the game state deduction / simulator when there is a hierarchical conflict;

[0010] The game state deduction / simulator of each layer is based on the confrontation state, unit action, object configuration and deduction strategy configuration structured data output by the structured processor of the game environment, selects actions from the executable action set to continuously deduce the future red-blue confrontation scenario, and outputs the deduction result, i.e. the situation to be evaluated, to the game state estimator, and combines the current situation score result output by the game state estimator to obtain the comprehensive score of the situation to be decided after selecting a specific action to be deduced to the future state, and outputs it to the multi-level value transfer and strategy generator; when there is a hierarchical conflict, it receives the comprehensive score of the upper and lower game problem situations transmitted by the multi-level value transfer and strategy generator, and calls the game state estimator to re-score the situation;

[0011] The multi-level value transfer and strategy generator is based on the number of confrontation levels output by the structured processor of the game environment, as well as the unit types and actions of each layer, to clarify the multi-level game confrontation sub-problems into which the complex game confrontation problem is decomposed, and to perform conflict judgment; when there is a hierarchical conflict, the comprehensive scores of the upper and lower game problems are passed to the game state deduction / simulator of this layer, and the comprehensive scores are re-obtained from the game state deduction / simulator of each layer to resolve the conflict, obtain a unified decision-making strategy, and output it to the strategy action conversion executor;

[0012] The strategy-action conversion executor converts the multi-level value transfer and decision-making strategies output by the strategy generator into actions that can be understood by the simulation environment according to the unit action structured data defined by the game environment structured processor, and outputs the results to the external simulation system.

[0013] Further, the game information processor includes a static knowledge building module and a dynamic information processing module;

[0014] The static knowledge construction module is used to build a knowledge base that supports the game confrontation process. Based on the information of domain experts, the static situation information system of the confrontation scenario with the same number and type of units as the object configuration scale is sorted and extracted. At the same time, the game level requirements and inter-level specifications are refined. The levels include global, local and individual, and the static knowledge information formed is output to the game state estimator.

[0015] The dynamic information processing module deeply processes the game information based on the confrontation status and unit action structured data, including the modeling of game unit movement behavior and future trends. The high-order behavior semantics of unit movement include: focus / assault / coordination / dispersion and outflanking, and outputs the formed dynamic processing information to the game state estimator.

[0016] Furthermore, the game state estimator and the game state deduction / simulator are provided with expert knowledge, tree search, deep neural network based on supervised learning technology and reinforcement learning network algorithm module library based on unsupervised learning; the tree structure deduction depth is determined according to the deduction method and parameters set by the deduction strategy configuration data, and the available algorithm modules are selected from the algorithm module library for combination and assembly, and finally the method parameters are configured to form a core solution module of a variety of game solution algorithms that conform to reinforcement learning, MCTS and rule decision-making;

[0017] When a hierarchical conflict occurs, the game state estimator corresponding to the conflicting level iteratively optimizes and trains the deep neural network of this layer based on the scoring results of the upper and lower layers.

[0018] Furthermore, the multi-level value transfer and strategy generator includes a hierarchical judgment module, a decision generation module, and a conflict resolution module;

[0019] The level judgment module is used to receive the object configuration structured data in the game environment structured processor, and judge the number of levels of the current red-blue confrontation problem based on the parameters. If the problem only contains one layer, the decision generation module is called; if the problem is divided into multiple layers, the conflict resolution module is called;

[0020] The decision generation module makes a final decision strategy based on the comprehensive score transmitted by the game state deduction / simulator, and transmits the final decision strategy to the strategy action conversion executor;

[0021] The conflict resolution module is used to receive the comprehensive scores of each layer of game problems transmitted by the game state deduction / simulator; when there is a conflict in the decision-making situation, the comprehensive scores of the upper and lower layer game problems are transmitted to the game state deduction / simulator of this layer, and the comprehensive scores of each layer are re-obtained to generate the decision strategy; finally, the calculated strategy is transmitted back to the strategy action conversion executor and fed back to the simulated or real game environment.

[0022] The present invention has the following beneficial effects:

[0023] 1. The multi-level game decision-making system proposed by the present invention, which integrates situational knowledge, can support large-scale heterogeneous intelligent group game deduction. After combining with the simulation training platform, it can meet the human-machine joint training and automated deduction of complex combat scenarios, greatly shortening the time for battlefield modeling and strategy solving.

[0024] 2. The present invention proposes a multi-level game decision-making system that integrates situational knowledge. By studying a modular, highly scalable common framework that integrates existing algorithms for solving game problems, different scenarios can be quickly modeled. According to the general game solving steps, appropriate algorithm solving strategies can be selected at each stage without repeatedly developing new systems, thereby achieving the purpose of reusing a set of frameworks for multiple scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a schematic diagram of the principle of a multi-level game decision-making system integrating situation knowledge according to the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0027] (1) The present invention is a multi-level game decision system that integrates situational knowledge. It designs standardized common abstract modules for game problems according to Boyd's cycle theory (observation, judgment, decision, and execution), and realizes the construction of game decision algorithms through module combination and assembly. First, the "observation" step is performed, and the game problem is sorted out and abstracted through the game environment structured processor; secondly, the "judgment" step is performed, and the game confrontation information is deeply mined and analyzed through the game information processor; then the "decision" step is performed, based on the processing results of the above two processors, the game state estimator and the game state deduction / simulator are used to deduce and score the situation, and the decision-making strategy is obtained by combining the decision generation module of the multi-level value transfer and strategy generator; finally, the "execution" step is performed, and the decision strategy is converted into instructions recognized by the external simulation system through the strategy action conversion executor;

[0028] (2) The system of the present invention conducts a common modeling of the game solving process based on the hierarchical concept. First, the sub-game problems and levels of the complex game problem are determined. Then, each level of the game problem follows the general architecture and makes decisions according to the process of (1). Finally, the conflict resolution module of the multi-level value transfer and strategy generator is used to resolve the multi-level conflicting decisions and obtain the final decision strategy. If the game problem does not contain sub-problems, the decision strategy can be obtained only through (1).

[0029] like Figure 1 As shown, a multi-level game decision-making system integrating situation knowledge specifically includes:

[0030] Multiple sub-game problem levels, each level includes a game environment structured processor, a game information processor, a game state estimator, and a game state deduction / simulator; and also includes a multi-level value transfer and strategy generator and a strategy action conversion executor;

[0031] The game environment structured processor of each layer is used to abstract the game information into various structured data that can be recognized and processed by the computer, including confrontation state, unit action, object configuration and deduction strategy configuration; the confrontation state structured data is output to the game information processor, game state estimator and game state deduction / simulator, and the confrontation state includes the traversable area, obstacles, high-value points, unit point information and unit execution action; the unit action structured data is output to the game information processor, game state deduction / simulator and strategy action conversion executor, and the unit action is formed by the executable actions of each unit in the confrontation scenario, including attack and movement; the object configuration structured data is output to the game information processor, game state estimator, game state deduction / simulator and multi-level value transfer and strategy generator, and the object configuration includes the number, type and confrontation level of each unit; the deduction strategy configuration structured data is output to the game state estimator and game state deduction / simulator, and the deduction strategy configuration includes the game method type and method parameters;

[0032] The game information processor at each layer is used to understand and identify the object configuration structured data, build a static situation information system in the fog environment, and analyze and recognize the dynamic processing information of the enemy unit based on the confrontation state structured data and unit action structured data sorted out by the game environment structured processor, including future positions, behaviors, intentions and threats, and output the analysis and cognition results including static knowledge and dynamic processing information to the game state estimator;

[0033] The game state estimator of each layer scores the deduction results output by the game state deduction / simulator based on the confrontation state, object configuration and deduction strategy configuration structured data provided by the game environment structured processor and the analysis and cognition results provided by the game information processor, evaluates the quality of action execution, and outputs the current situation score result to the game state deduction / simulator; it is also used to re-score the deduction results output by the game state deduction / simulator when there is a hierarchical conflict;

[0034] The game state deduction / simulator of each layer is based on the confrontation state, unit action, object configuration and deduction strategy configuration structured data output by the structured processor of the game environment, selects actions from the executable action set to continuously deduce the future red-blue confrontation scenario, and outputs the deduction result, i.e. the situation to be evaluated, to the game state estimator, and combines the current situation score result output by the game state estimator to obtain the comprehensive score of the situation to be decided after selecting a specific action to be deduced to the future state, and outputs it to the multi-level value transfer and strategy generator; when there is a hierarchical conflict, it receives the comprehensive score of the upper and lower game problem situations transmitted by the multi-level value transfer and strategy generator, and calls the game state estimator to re-score the situation;

[0035] The multi-level value transfer and strategy generator is based on the number of confrontation levels output by the structured processor of the game environment, as well as the unit types and actions of each layer, to clarify the multi-level game confrontation sub-problems into which the complex game confrontation problem is decomposed, and to perform conflict judgment; when there is a hierarchical conflict, the comprehensive score of the upper-level game problem situation is passed to the game state deduction / simulator of the lower layer, and the comprehensive score is re-obtained from the game state deduction / simulator of each layer to resolve the conflict, obtain a unified decision-making strategy, and output it to the strategy action conversion executor;

[0036] The strategy-action conversion executor converts the multi-level value transfer and decision-making strategies output by the strategy generator into actions that can be understood by the simulation environment according to the unit action structured data defined by the game environment structured processor, and outputs the results to the external simulation system.

[0037] Among them, the game information processor includes a static knowledge building module and a dynamic information processing module;

[0038] The static knowledge construction module is used to build a knowledge base that supports the game confrontation process. Based on the information of domain experts, the static situation information system of the confrontation scenario with the same number and type of units as the object configuration scale is sorted and extracted. At the same time, the game level requirements and inter-level specifications are refined. The levels include global, local and individual, and the static knowledge information formed is output to the game state estimator.

[0039] The dynamic information processing module deeply processes the game information based on the confrontation status and unit action structured data, including the modeling of game unit movement behavior and future trends. The high-order behavior semantics of unit movement include: focus / assault / coordination / dispersion and outflanking, and outputs the formed dynamic processing information to the game state estimator.

[0040] Among them, the game state estimator and game state deduction / simulator are equipped with expert knowledge, tree search, deep neural network based on supervised learning technology and reinforcement learning network algorithm module library based on unsupervised learning; the deduction depth of the tree structure is determined according to the deduction method and parameters set by the deduction strategy configuration data, and the available algorithm modules are selected from the algorithm module library for combination and assembly. Finally, the method parameters are configured to form a core solution module that conforms to various game solution algorithms such as reinforcement learning, MCTS and rule decision-making;

[0041] When a hierarchical conflict occurs, the game state estimator corresponding to the conflicting level iteratively optimizes and trains the deep neural network of this layer based on the scoring results of the upper and lower layers to improve the valuation effect and effectively avoid hierarchical decision conflicts caused by inaccurate valuations.

[0042] Among them, the multi-level value transfer and strategy generator includes a hierarchical judgment module, a decision generation module, and a conflict resolution module;

[0043] The level judgment module is used to receive the object configuration structured data in the game environment structured processor, and judge the number of levels of the current red-blue confrontation problem based on the parameters. If the problem only contains one layer, the decision generation module is called; if the problem is divided into multiple layers, the conflict resolution module is called;

[0044] The decision generation module makes a final decision strategy based on the comprehensive score transmitted by the game state deduction / simulator, and transmits the final decision strategy to the strategy action conversion executor;

[0045] The conflict resolution module is used to receive the comprehensive scores of each layer of game problems transmitted by the game state deduction / simulator; when there is a conflict in the decision-making situation, the comprehensive scores of the upper and lower layer game problems are transmitted to the game state deduction / simulator of this layer, and the comprehensive scores of each layer are re-obtained to generate the decision strategy; finally, the calculated strategy is transmitted back to the strategy action conversion executor and fed back to the simulated or real game environment.

[0046] The following is an example of 9*9 Gobang:

[0047] In the first step, the action, state, etc. are structured through the game environment structured processor, and the level of the current problem is determined. Since the action space of the 9*9 chessboard is not large, it does not need to be decomposed again, so the level is set to 1. The action can be structured as {"1 row and 1 column", ..., "1 row and 9 columns", ..., "n rows and m columns", ..., "9 rows and 9 columns"}, and the state can be structured as {"There is 1 chess piece on the board, 1 row and 1 column is a black piece", ..., "There are 81 chess pieces on the board, 1 row and 1 column is a black piece, ..."}. The structured game information is abstracted into various structured data that are easier for computers to recognize and process through matrices, dictionaries, etc.

[0048] In the second step, the game information processor can introduce data analysis algorithms such as intention prediction to predict where the opponent will place his move on the chessboard, assist in subsequent decision-making, and achieve secondary abstraction and cognition of structured information.

[0049] In the third step, the game state estimator constructs a valuation method for the current state and the current action in a plug-and-play manner, and determines the value score of the chess piece after it is moved to different positions (performing different actions) and becomes different states. This can be done through various methods such as reinforcement learning, supervised learning, transfer learning, and expert knowledge.

[0050] The fourth step is to use the game state deduction / simulator based on the Gobang rules. The rules of this confrontation process include: only one piece can be placed in each position.

[0051] The third and fourth steps are the core of this architecture. Various logical rules and algorithms are set through modules, parameters, etc. to form different types of game decision-making methods, and generate decisions based on optimal valuation. For example, when the deduction depth is set to 1, the system directly obtains the comprehensive valuation of the future situation of various actions under the current state, which is a typical reinforcement learning decision-making process. When the deduction depth is set to 5, the system will construct the deduction simulation results of the future state through trees, graphs, etc., which is another process of making decisions by tree search or graph search. For the valuation of simulation results, if new methods such as deep learning and reinforcement learning are used, it becomes a decision-making framework similar to AlphaGo. Otherwise, if expert rules or the valuation module of the UCT (UpperConfidential Bound on Tree) algorithm are used, it is a classic heuristic function valuation.

[0052] In the fifth step, the calculation results of each layer are passed to the multi-level value transmitter. The upper and lower layer strategies are transmitted in the form of weighted benefits, which affects the valuation function or benefit function of this layer. Each level communicates with each other through the multi-level value transfer module to form a complete common framework for multi-layer heterogeneous intelligent agent collaborative game. However, since the level of this problem is 1, no transmission is required.

[0053] The sixth step is to pass the calculated strategy back to the strategy action conversion executor so that it can be fed back to the simulation or real game environment.

[0054] The system proposed by this method is a standardized and modular game decision system, which standardizes and abstracts game problems and supports flexible access and deletion of each module. In the example of Gobang, if there is no need to perform secondary abstraction and cognition on the structured information, the game information processor can be processed without any processing, and the input of the game information processor is directly passed to the next module as its output; in terms of the selection of game state estimators, it supports the access and configuration of multiple algorithms, including expert knowledge, tree search, and reinforcement learning, so as to facilitate the migration of solved game decision algorithms (game decision solution methods for old problems) to new game problem solution algorithms (game solution methods to be designed). This method can quickly model the confrontation environment in different scenarios, and select the appropriate algorithm solution strategy at each stage according to the general game solution steps without repeatedly developing new systems, so as to achieve a set of frameworks for multiple scenarios.

[0055] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A multi-level game decision system integrating situation knowledge, characterized in that: It includes multiple sub-game problem levels, each of which includes a game environment structured processor, a game information processor, a game state estimator, and a game state deduction / simulator; and also includes a multi-level value transfer and strategy generator and a strategy action conversion executor; The game environment structured processor of each layer is used to abstract the game information into various structured data that can be recognized and processed by the computer, including confrontation state, unit action, object configuration and deduction strategy configuration; the confrontation state structured data is output to the game information processor, game state estimator and game state deduction / simulator, and the confrontation state includes the traversable area, obstacles, high-value points, unit point information and unit execution action; the unit action structured data is output to the game information processor, game state deduction / simulator and strategy action conversion executor, and the unit action is formed by the executable actions of each unit in the confrontation scenario, including attack and movement; the object configuration structured data is output to the game information processor, game state estimator, game state deduction / simulator and multi-level value transfer and strategy generator, and the object configuration includes the number, type and confrontation level of each unit; the deduction strategy configuration structured data is output to the game state estimator and game state deduction / simulator, and the deduction strategy configuration includes the game method type and method parameters; The game information processor at each layer is used to understand and identify the object configuration structured data, build a static situation information system in the fog environment, and analyze and recognize the dynamic processing information of the enemy unit based on the confrontation state structured data and unit action structured data sorted out by the game environment structured processor, including future positions, behaviors, intentions and threats, and output the analysis and cognition results including static knowledge and dynamic processing information to the game state estimator; The game state estimator of each layer scores the deduction results output by the game state deduction / simulator based on the confrontation state, object configuration and deduction strategy configuration structured data provided by the game environment structured processor and the analysis and cognition results provided by the game information processor, evaluates the quality of action execution, and outputs the current situation score result to the game state deduction / simulator; It is also used to re-score the game state simulation / simulator output results when there is a hierarchical conflict; The game state deduction / simulator of each layer is based on the confrontation state, unit action, object configuration and deduction strategy configuration structured data output by the structured processor of the game environment, selects actions from the executable action set to continuously deduce the future red-blue confrontation scenario, and outputs the deduction result, i.e. the situation to be evaluated, to the game state estimator, and combines the current situation score result output by the game state estimator to obtain the comprehensive score of the situation to be decided after selecting a specific action to be deduced to the future state, and outputs it to the multi-level value transfer and strategy generator; when there is a hierarchical conflict, it receives the comprehensive score of the upper and lower game problem situations transmitted by the multi-level value transfer and strategy generator, and calls the game state estimator to re-score the situation; The multi-level value transfer and strategy generator is based on the number of confrontation levels output by the structured processor of the game environment, as well as the unit types and actions of each layer, to clarify the multi-level game confrontation sub-problems into which the complex game confrontation problem is decomposed, and to perform conflict judgment; when there is a hierarchical conflict, the comprehensive scores of the upper and lower game problems are passed to the game state deduction / simulator of this layer, and the comprehensive scores are re-obtained from the game state deduction / simulator of each layer to resolve the conflict, obtain a unified decision-making strategy, and output it to the strategy action conversion executor; The strategy-action conversion executor converts the multi-level value transfer and decision-making strategies output by the strategy generator into actions that can be understood by the simulation environment according to the unit action structured data defined by the game environment structured processor, and outputs the results to the external simulation system.

2. A multi-level game decision system integrating situation knowledge according to claim 1, characterized in that: The game information processor includes a static knowledge building module and a dynamic information processing module; The static knowledge construction module is used to build a knowledge base that supports the game confrontation process. Based on the information of domain experts, the static situation information system of the confrontation scenario with the same number and type of units as the object configuration scale is sorted and extracted. At the same time, the game level requirements and inter-level specifications are refined. The levels include global, local and individual, and the static knowledge information formed is output to the game state estimator. The dynamic information processing module deeply processes the game information based on the confrontation status and unit action structured data, including the modeling of game unit movement behavior and future trends. The high-order behavior semantics of unit movement include: focus / assault / coordination / dispersion and outflanking, and outputs the formed dynamic processing information to the game state estimator.

3. A multi-level game decision system integrating situation knowledge according to claim 1, characterized in that: The game state estimator and game state deduction / simulator are equipped with expert knowledge, tree search, deep neural network based on supervised learning technology and reinforcement learning network algorithm module library based on unsupervised learning; the tree structure deduction depth is determined according to the deduction method and parameters set by the deduction strategy configuration data, and the available algorithm modules are selected from the algorithm module library for combination and assembly. Finally, the method parameters are configured to form a core solution module that conforms to various game solution algorithms such as reinforcement learning, MCTS and rule decision-making; When a hierarchical conflict occurs, the game state estimator corresponding to the conflicting level iteratively optimizes and trains the deep neural network of this layer based on the scoring results of the upper and lower layers.

4. A multi-level game decision system integrating situational knowledge according to claim 1, characterized in that: The multi-level value transfer and strategy generator includes a hierarchical judgment module, a decision generation module, and a conflict resolution module; The level judgment module is used to receive the object configuration structured data in the game environment structured processor, and judge the number of levels of the current red-blue confrontation problem based on the parameters. If the problem only contains one layer, the decision generation module is called; if the problem is divided into multiple layers, the conflict resolution module is called; The decision generation module makes a final decision strategy based on the comprehensive score transmitted by the game state deduction / simulator, and transmits the final decision strategy to the strategy action conversion executor; The conflict resolution module is used to receive the comprehensive scores of each layer of game problems transmitted by the game state deduction / simulator; When there is a conflict in the decision-making situation, the comprehensive scores of the upper and lower game problem situations are passed to the game state deduction / simulator of this layer, and the comprehensive scores of the situations at each layer are re-obtained to generate the decision-making strategy; finally, the calculated strategy is passed back to the strategy action conversion executor and fed back to the simulation or real game environment.

Images