Multistage deduction interaction data processing method and system based on intelligent game

Abstract

The invention discloses a multi-level deduction interaction data processing method and system based on intelligent game. The system comprises a user development component used for configuring basic parameters of the system, controlling the game process and providing a visual operation interface; the third-party environment access component is used for providing an information interface for accessing a third-party environment; the agent service component is used for packaging agent game information and storing the agent game information to the database component; the multi-stage deduction assembly comprises an upper-stage deduction action set, a middle-stage deduction action set and a lower-stage deduction action set to complete multi-stage deduction. Components are clear, modularized labor division is achieved, implementation is convenient, switching can be conducted under the single-agent and multi-agent conditions, resource waste is reduced, intelligent gaming can be conducted in a targeted mode through three strategies, free switching is conducted under the same and different environment conditions, the analysis efficiency is improved, and the intelligent gaming capacity and efficiency are higher than those of an existing model.

Description

Technical Field

[0001] The present invention relates to the field of intelligent agent intelligent game technology, and in particular to a multi-level deduction interactive data processing method and system based on intelligent game. Background Art

[0002] The development of artificial intelligence technology has gone through three stages: computational intelligence, perceptual intelligence, and cognitive intelligence. Computational intelligence mainly involves rule-based operations such as scientific operations, logical processing, and statistical queries; perceptual intelligence is represented by image understanding, speech recognition, and machine translation, and is based on deep learning models; cognitive intelligence is represented by understanding, reasoning, thinking, and decision-making, emphasizing cognitive reasoning and autonomous learning capabilities. Intelligent game, as a cutting-edge paradigm of decision-making intelligence, is a high-level manifestation of cognitive intelligence. It is mainly based on game theory, emphasizing the inclusion of other intelligent agents (teammates and opponents) in one's own decision-making for rule self-learning, game confrontation evolution, and explainable strategy recommendation. The development of artificial intelligence has created new possibilities in the field of game theory. At the theoretical level, artificial intelligence provides new solutions to games, such as Nash equilibrium and equilibrium in cooperative games. In practical applications, artificial intelligence has also been applied in new game scenarios such as Go and games. For example, AlphaGo and AlphaStar have obtained excellent strategies through the application of deep reinforcement learning methods and defeated top human players in human-computer game confrontation.

[0003] Chinese invention patent CN112295229B discloses an intelligent game confrontation platform, including a simulation deduction platform and an AI confrontation training environment management subsystem. From the aspects of system functional composition, interaction relationship and workflow, a system design method for an intelligent game confrontation platform is proposed, aiming to solve the problems of environmental simulation and test system construction for intelligent game confrontation training tests. The simulation deduction platform provides functions such as battlefield environment simulation, intelligence simulation, weapon platform simulation, damage simulation and real-time intelligent command intervention; the AI ​​confrontation training environment management subsystem configures operating environment parameters and operating condition parameters, and uses container technology to create a training confrontation platform from the simulation deduction platform image, dynamically forming a prototype of the confrontation training test environment. The main focus of this invention is on enhancing environmental adaptability, and it can be used as a plug-in for other intelligent game platforms, but the main body logic is low and the performance improvement is small.

[0004] Chinese invention patent CN113379054A is an open intelligent game ecological platform, in which the platform includes a central site and at least one confrontation system sub-site, each confrontation system sub-site operates independently to realize the confrontation function; the central site interacts with the sub-site data interface of each confrontation system sub-site through the central data interface; when the sub-site data interface of any confrontation system sub-site is opened, the central site transmits the user information of the transferred user to any confrontation system sub-site, and any confrontation system sub-site transmits the sub-site confrontation data to the central site; wherein the transferred user is a user who logs in to any confrontation system sub-site through the central site. The idea of ​​this invention patent is to use distributed computing functions to improve the overall computing efficiency, which is an idea to improve computing speed, but the improvement in the real intelligent game logic is still very small.

[0005] The methods and approaches provided in the above invention patents do not specifically and in-depth explain how to improve the game's ability in the current intelligent game process, nor do they involve ways to reuse existing experience in multiple intelligent games. Therefore, the logical architecture of existing intelligent games is not ideal. Summary of the invention

[0006] The present invention provides a multi-level deduction interactive data processing method and system based on intelligent game, so as to solve the technical problems of unclear logic, unclear multi-level deduction process and unsatisfactory utilization of existing confrontation analysis experience in the existing intelligent game process.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: The present invention provides a multi-level deduction interactive data processing method and system based on intelligent game, comprising: A control unit and an intelligent game unit, wherein the control unit includes a user development component and a third-party environment access component, and the intelligent game unit includes an agent service component, a multi-level deduction component, a database component, and a system background component; The user development component is used to configure the basic parameters of the system, control the game process, and provide a visual operation interface; The third-party environment access component is used to provide an information interface for accessing a third-party environment, the user development component selects a third-party environment to access, and the third-party environment access component provides the database component with third-party environment access component information; The agent service component includes a single agent service set and a multi-agent service set, the agent service component is used to encapsulate agent game information, the agent game information is encapsulated as an experience information set and stored in the database component, and the agent service component receives multi-level deduction data from the multi-level deduction component; The multi-level deduction component includes an upper-level deduction action set, an intermediate deduction action set and a lower-level deduction action set, and the upper-level deduction action set, the intermediate deduction action set and the lower-level deduction action set complete multi-level deduction through deduction actions, and the multi-level deduction component generates the multi-level deduction data and sends it to the intelligent agent service component; The database component receives the third-party environment access component information from the third-party environment access component, and the database component receives and provides the experience information set from the agent service component; The system background component provides interface and routing services to connect the user development component, third-party environment access component, intelligent agent service component, multi-level deduction component, and database component to the network.

[0008] Further, the user development component is deployed on a single control computer; The user development component can adjust the configuration parameters of the agent service component to select and use a single-agent service set and a multi-agent service set; The user development component selects a third-party environment accessed in the third-party environment access component; The user development component views the operation and maintenance status information through a visual operation interface.

[0009] Furthermore, the third-party environment access component can connect to multiple third-party environments; The third-party environment is an existing simulation operation platform; The third-party environment access component stores the used existing simulation operation platform in the database component.

[0010] Further, the agent service component includes a single-agent service set and a multi-agent service set; The single-agent service set copies the experience information set in the database component into experience information 1 and experience information 2, optimizes experience information 1 using policy gradient optimization to obtain a policy network, optimizes experience information 2 using value gradient optimization to obtain a value network, and obtains deterministic actions from the policy network and sends them to the value network to predict the single-agent action set; The single agent training process is as follows: the control game engine starts, the single agent takes action, obtains and sends the current state, evaluates the situation and calculates the decision, on the one hand, continues to obtain the situation, on the other hand, uses the current state to fill in the experience information, infers the state change to update the parameters, and determines whether the state change expectation converges. If not, return to the inference state change to perform the parameter update step. If converged, store the experience information, and store the experience information in the database together with the other hand to determine whether the game result meets the expectation. The multi-agent service set shares the experience information set in the database component with all agents in the multi-agent service set to implement parallel execution of the experience information set strategy; The single-agent service set and the multi-agent service set process the experience information to obtain game information, and then encapsulate and store it in the database component.

[0011] Further, the multi-level deduction component obtains observation information by observing the intelligent agent service component; The observation information includes campaign-level observation information and tactical-level observation information.

[0012] Furthermore, the multi-level deduction process of the multi-level deduction component is as follows: In the multi-level deduction process 1, the upper-level deduction action set generates a campaign-level decision through a pulse neural network based on a monotone value function decomposition algorithm, and obtains an upper-level deduction action strategy and sends it to the intermediate deduction action set; In the second multi-level deduction process, the intermediate deduction action level generates tactical-level decisions through a pulse neural network based on a monotone value function decomposition algorithm, and obtains an intermediate deduction action strategy which is sent to a lower-level deduction action set; In multi-level deduction process three, the lower-level deduction action level generates tactical-level decisions through a pulse neural network based on a proximal strategy optimization algorithm.

[0013] Further, the database component receives one third-party environment at a time when receiving the third-party environment access component information; The experience information set includes a training experience information set and a testing experience information set.

[0014] Furthermore, the training information set and the test experience information set generation process are as follows: When the database component currently uses the third-party environment for the first time, the intelligent agent service component receives the experience information set, which is a test experience information set. The intelligent agent service component processes the test experience information set, encapsulates it, and sends it to the database component to obtain a training experience information set. When the database component continues to use the third-party environment, the intelligent agent service component obtains the training experience information set from the database component, processes and encapsulates it to obtain a new training experience information set and sends it to the database component.

[0015] Furthermore, it also includes a viewing component that can record and store the multi-level deduction process and the experience information, and realize playback and double-speed playback according to the user-developed component.

[0016] Furthermore, the system background component simultaneously controls user access to the parameter control unit, generates operation and maintenance status information, and maintains the smooth operation of the system.

[0017] The beneficial effects brought about by the technical solution provided by the present invention include at least: The present invention provides a multi-level deduction interactive data processing method and system based on intelligent game, including: a parameter control unit and an intelligent game unit, the parameter control unit includes a user development component and a third-party environment access component, the intelligent game unit includes an agent service component, a multi-level deduction component, a database component, and a system background component; the user development component is used for basic parameter configuration of the system, controlling the game process, and providing a visual operation interface; the third-party environment access component is used to provide an information interface for accessing a third-party environment, the user development component selects the third-party environment to be accessed, and the third-party environment access component provides the database component with third-party environment access component information; the agent service component includes a single-agent service set and a multi-agent service set, the agent service component is used to encapsulate agent game information, and the The agent game information is encapsulated as an experience information set and stored in the database component. The agent service component receives multi-level deduction data from the multi-level deduction component. The multi-level deduction component includes an upper-level deduction action set, an intermediate deduction action set and a lower-level deduction action set. The upper-level deduction action set, the intermediate deduction action set and the lower-level deduction action set complete multi-level deduction through deduction actions. The multi-level deduction component generates the multi-level deduction data and sends it to the agent service component. The database component receives the third-party environment access component information from the third-party environment access component. The database component receives and provides the experience information set from the agent service component. The system background component provides an interface and routing service to connect the user development component, the third-party environment access component, the agent service component, the multi-level deduction component and the database component to a network. The components of the present invention are clear and modular, and the division of labor is easy to implement.

[0018] In the present invention, the user development component can adjust the configuration parameters of the agent service component, so as to choose to use a single-agent service set and a multi-agent service set, and can switch between single-agent and multi-agent situations to reduce resource waste.

[0019] The multi-level deduction component in the present invention includes a three-level deduction process, which can use three strategies for high and low levels to carry out intelligent games in a targeted manner.

[0020] The present invention distinguishes between the generation of training information sets and test experience information sets, and can switch freely when using the same or different environments, thereby improving analysis efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 It is a structural schematic diagram of a multi-level deduction interactive data processing method and system based on intelligent game provided by the present invention; Figure 2 This is a flow chart of the single agent training process of the present invention; Figure 3 It is a schematic diagram of the structure of the multi-level deduction component of the present invention. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Example

[0024] This embodiment provides a multi-level deduction interactive data processing method and system based on intelligent game, including: a parameter control unit and an intelligent game unit, the parameter control unit includes a user development component and a third-party environment access component, the intelligent game unit includes an intelligent agent service component, a multi-level deduction component, a database component, and a system background component; The user development component is used to configure the basic parameters of the system, control the game process, and provide a visual operation interface; The third-party environment access component is used to provide an information interface for accessing a third-party environment, the user development component selects a third-party environment to access, and the third-party environment access component provides the database component with third-party environment access component information; The agent service component includes a single agent service set and a multi-agent service set, the agent service component is used to encapsulate agent game information, the agent game information is encapsulated as an experience information set and stored in the database component, and the agent service component receives multi-level deduction data from the multi-level deduction component; The multi-level deduction component includes an upper-level deduction action set, an intermediate deduction action set and a lower-level deduction action set, and the upper-level deduction action set, the intermediate deduction action set and the lower-level deduction action set complete multi-level deduction through deduction actions, and the multi-level deduction component generates the multi-level deduction data and sends it to the intelligent agent service component; The database component receives the third-party environment access component information from the third-party environment access component, and the database component receives and provides the experience information set from the agent service component; The system background component provides interface and routing services to connect the user development component, third-party environment access component, intelligent agent service component, multi-level deduction component, and database component to the network.

[0025] Specifically, the user development component is deployed on a single control computer; The user development component can adjust the configuration parameters of the agent service component to select and use a single-agent service set and a multi-agent service set; The user development component selects a third-party environment accessed in the third-party environment access component; The user development component views the operation and maintenance status information through a visual operation interface.

[0026] Please refer to Figure 1 The structure diagram of the multi-level deduction interactive data processing method and system based on intelligent game provided by the present invention is shown.

[0027] Specifically, the third-party environment access component can connect to multiple third-party environments; The third-party environment is an existing simulation operation platform; The third-party environment access component stores the used existing simulation operation platform in the database component.

[0028] Specifically, the agent service component includes a single-agent service set and a multi-agent service set; The single-agent service set copies the experience information set in the database component into experience information 1 and experience information 2, optimizes experience information 1 using policy gradient optimization to obtain a policy network, optimizes experience information 2 using value gradient optimization to obtain a value network, and obtains deterministic actions from the policy network and sends them to the value network to predict the single-agent action set; Please refer to Figure 2 The flowchart of the single agent training process of the present invention is shown.

[0029] The single agent training process is as follows: the control game engine starts, the single agent takes action, obtains and sends the current state, evaluates the situation and calculates the decision, on the one hand, continues to obtain the situation, on the other hand, uses the current state to fill in the experience information, infers the state change to update the parameters, and determines whether the state change expectation converges. If not, return to the inference state change to perform the parameter update step. If converged, store the experience information, and store the experience information in the database together with the other hand to determine whether the game result meets the expectation. The multi-agent service set shares the experience information set in the database component with all agents in the multi-agent service set to implement parallel execution of the experience information set strategy; The single-agent service set and the multi-agent service set process the experience information to obtain game information, and then encapsulate and store it in the database component.

[0030] Specifically, the multi-level deduction component obtains observation information by observing the intelligent agent service component; The observation information includes campaign-level observation information and tactical-level observation information.

[0031] Specifically, the multi-level deduction process of the multi-level deduction component is: In the multi-level deduction process 1, the upper-level deduction action set generates a campaign-level decision through a pulse neural network based on a monotone value function decomposition algorithm, and obtains an upper-level deduction action strategy and sends it to the intermediate deduction action set; In the second multi-level deduction process, the intermediate deduction action level generates tactical-level decisions through a pulse neural network based on a monotone value function decomposition algorithm, and obtains an intermediate deduction action strategy which is sent to a lower-level deduction action set; In multi-level deduction process three, the lower-level deduction action level generates tactical-level decisions through a pulse neural network based on a proximal strategy optimization algorithm.

[0032] Please refer to Figure 3 The schematic diagram of the structure of the multi-level deduction component of the present invention is shown.

[0033] Specifically, the database component receives one third-party environment at a time when receiving the third-party environment access component information; The experience information set includes a training experience information set and a testing experience information set.

[0034] Specifically, the training information set and the test experience information set generation process are as follows: When the database component currently uses the third-party environment for the first time, the intelligent agent service component receives the experience information set, which is a test experience information set. The intelligent agent service component processes the test experience information set, encapsulates it, and sends it to the database component to obtain a training experience information set. When the database component continues to use the third-party environment, the intelligent agent service component obtains the training experience information set from the database component, processes and encapsulates it to obtain a new training experience information set and sends it to the database component.

[0035] Specifically, the system also includes a viewing component that can record and store the multi-level deduction process and the experience information, and realize playback and double-speed playback according to the user-developed component.

[0036] Specifically, the system background component simultaneously controls user access to the parameter control unit, generates operation and maintenance status information, and maintains the smooth operation of the system. Example

[0037] This embodiment provides a multi-level deduction interactive data processing method and system based on intelligent game, which is specifically: In this embodiment, the Warcraft 3 engine produced by Blizzard, version 1.32.10, AI enhanced AMAI version 3.3.1 is used. Warcraft 3 is a classic real-time strategy game, and its battle mode and AI design have been tested for more than 20 years. Countless artificial intelligence testers have tested through this game.

[0038] AMAI (Advanced Melee AI) is a well-known open source AI enhancement MOD in Warcraft 3. It completely rewrites the game's AI script, making the AI ​​have behavior patterns that are closer to human players, including tactics, strategies, and operations.

[0039] The manual development component in this embodiment is actually a computer work platform. It sends an environment selection command to the third-party environment access component, selects Warcraft 3, version 1.32.10, AI enhanced AMAI3.3.1 version in the third-party environment component, and selects the regular map TurtleRock as the third-party environment.

[0040] At this time, the third-party environment access component provides the third-party environment access component information to the database component. In this embodiment, the existing built-in enhanced version of AI AMAI3.3.1 is stored in the database component through the third-party environment component as experience information.

[0041] At this time, for a single agent, for a single unit in the battle map, when a single operation instruction or AI default instruction is given, the single agent service set is selected. For a single agent in Warcraft 3, regular units have 12 operation instructions and 6 item instructions.

[0042] The single-agent service set copies the AMAI3.3.1 experience information set in the database component into experience information 1 and experience information 2, uses policy gradient optimization to optimize experience information 1 to obtain a policy network, and uses value gradient optimization to optimize experience information 2 to obtain a value network. The deterministic actions obtained by the policy network are sent to the value network to predict the single-agent action set, thereby realizing the behavioral operations of a single agent in simple actions.

[0043] The single-agent training process is as follows: the control game engine starts, the single-agent takes action, obtains and sends the current state, evaluates the situation and calculates the decision, on the one hand, continues to obtain the situation, on the other hand, uses the current state to fill in the experience information, infers the state change to update the parameters, and determines whether the state change expectation converges. If not, return to the inference state change to perform the parameter update step. If converged, store the experience information, and on the other hand, store the experience information in the database together to determine whether the game result meets expectations.

[0044] For multiple intelligent agents, centralized training and multiple agent execution behaviors are used. The training experience of each unit is shared with all units. When the number of units is large, the above operation instructions can be executed in parallel, thereby accelerating the learning process.

[0045] The single-agent service set and the multi-agent service set process the experience information to obtain the game information, and then encapsulate and store it in the database component.

[0046] In this embodiment, the main deduction process of the multi-level deduction component is divided into three levels.

[0047] In the multi-level deduction process 1, the upper-level deduction action set generates a campaign-level decision through a pulse neural network based on a monotone value function decomposition algorithm, and obtains an upper-level deduction action strategy and sends it to the intermediate deduction action set; In the second multi-level deduction process, the intermediate deduction action level generates tactical-level decisions through a pulse neural network based on a monotone value function decomposition algorithm, and obtains an intermediate deduction action strategy which is sent to a lower-level deduction action set; In multi-level deduction process three, the lower-level deduction action level generates tactical-level decisions through a pulse neural network based on a proximal strategy optimization algorithm.

[0048] The upper-level deduction action set is used to analyze and implement the global strategy in the battle process. For existing models, including but not limited to: 1. Fast Attack Tactics: This strategy involves sacrificing economic and technological progress to quickly build an army in the early stages of the game to defeat the opponent. In order to speed up this process, players often need to secretly build military buildings near the enemy base and may mobilize all peasants to participate in the attack.

[0049] 2. One-wave tactics: By predicting the moments when the enemy's forces are weak and adopting specific operational strategies, assembling the strongest possible army to launch a general offensive at these critical moments, which is usually accompanied by the timely upgrade of key technologies.

[0050] 3. Suppression tactics: Send troops forward to force the opponent into a disadvantageous position or limit its expansion, while using this opportunity to expand yourself.

[0051] 4. Harassment tactics: Use highly mobile units, air units or transport units to avoid the enemy's main force and directly attack the enemy's peasants to weaken the enemy's economic base.

[0052] 5. Map Control: When the two sides are evenly matched, players move flexibly on the battlefield and conduct extensive reconnaissance to understand the location and composition of the enemy's main force, waiting for the right time to attack or harass.

[0053] 6. Stealing economy and technology: When you think that the enemy will not conduct detailed reconnaissance, you can take the risk of opening a branch mine in a remote area to gain economic advantage, or sacrifice manpower or economy to develop key technology in advance, thereby gaining the initiative to attack.

[0054] 7. Late-game tactics: In the early and middle stages of the game, the focus is on defense and expansion, and a large number of defensive buildings are built to stabilize the front line. With the support of economy and technology, a large number of advanced units are eventually produced to gradually consume and eliminate the opponent.

[0055] 8. Switching base tactics: When you are not as strong as your opponent in frontal combat, use guerrilla tactics to avoid the enemy's main force, and when the enemy attacks, go around to their base to carry out mutual destruction.

[0056] The intermediate deduction action set is mainly aimed at the strategy generation generated in a smaller-scale battle, such as the strategy generation in the early stage of jungle when there are fewer units.

[0057] The monotone function decomposition method Qmix is ​​a classic algorithm in multi-agent reinforcement learning (MARL). It has made some improvements based on VDN (Value-Decomposition Networks). The main advantage of QMIX is that it performs better in environments where there are large differences between agents, can better handle the interaction and cooperation problems between agents, and can achieve a strategy of changing exploration to achieve global result optimization under conditions of less information interaction by relying on reward mechanisms.

[0058] Spiking neural network (SNN) is a commonly used network model in military command, strategy optimization and other fields. SNN processes information by simulating the pulse emission behavior of biological neurons. The neurons in SNN will emit pulses when they receive sufficient input instead of continuously outputting activation values. It can have higher efficiency and energy efficiency when processing sudden events and urgent tasks such as games and military operations.

[0059] The lower-level deduction action level generates tactical-level decisions through a pulse neural network based on the proximal policy optimization algorithm. In this embodiment, the lower-level deduction action is generated for the specific actions of a specific unit in the scene. The core idea of ​​the proximal policy optimization algorithm PPO is to limit the difference between the new policy and the old policy each time the policy is updated to maintain the stability of the policy. Specifically, PPO introduces a "replacement" objective function that takes into account the difference between the new and old policies at each update to ensure that the new policy does not deviate too far from the old policy.

[0060] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0061] The embodiments of the present invention are described with reference to flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0062] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0063] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or still includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0064] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for those skilled in the art, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A multi-level deduction interactive data processing method and system based on intelligent game, characterized in that: include: A control unit and an intelligent game unit, wherein the control unit includes a user development component and a third-party environment access component, and the intelligent game unit includes an agent service component, a multi-level deduction component, a database component, and a system background component; The user development component is used to configure the basic parameters of the system, control the game process, and provide a visual operation interface; The third-party environment access component is used to provide an information interface for accessing a third-party environment, the user development component selects a third-party environment to access, and the third-party environment access component provides the database component with third-party environment access component information; The agent service component includes a single agent service set and a multi-agent service set, the agent service component is used to encapsulate agent game information, the agent game information is encapsulated as an experience information set and stored in the database component, and the agent service component receives multi-level deduction data from the multi-level deduction component; The multi-level deduction component includes an upper-level deduction action set, an intermediate deduction action set and a lower-level deduction action set, and the upper-level deduction action set, the intermediate deduction action set and the lower-level deduction action set complete multi-level deduction through deduction actions, and the multi-level deduction component generates the multi-level deduction data and sends it to the intelligent agent service component; The database component receives the third-party environment access component information from the third-party environment access component, and the database component receives and provides the experience information set from the agent service component; The system background component provides interface and routing services to connect the user development component, third-party environment access component, intelligent agent service component, multi-level deduction component, and database component to the network.

2. According to claim 1, a multi-level deduction interactive data processing method and system based on intelligent game, characterized in that: The user developed component is deployed on a single control computer; The user development component can adjust the configuration parameters of the agent service component to select and use a single-agent service set and a multi-agent service set; The user development component selects a third-party environment accessed in the third-party environment access component; The user development component views the operation and maintenance status information through a visual operation interface.

3. According to claim 1, a multi-level deduction interactive data processing method and system based on intelligent game, characterized in that: The third-party environment access component can connect to multiple third-party environments; The third-party environment is an existing simulation operation platform; The third-party environment access component stores the used existing simulation operation platform in the database component.

4. According to claim 1, a multi-level deduction interactive data processing method and system based on intelligent game, characterized in that: The agent service component includes a single-agent service set and a multi-agent service set; The single-agent service set copies the experience information set in the database component into experience information 1 and experience information 2, optimizes experience information 1 using policy gradient optimization to obtain a policy network, optimizes experience information 2 using value gradient optimization to obtain a value network, and obtains deterministic actions from the policy network and sends them to the value network to predict the single-agent action set; The single agent training process is as follows: the control game engine starts, the single agent takes action, obtains and sends the current state, evaluates the situation and calculates the decision, on the one hand, continues to obtain the situation, on the other hand, uses the current state to fill in the experience information, infers the state change to update the parameters, and determines whether the state change expectation converges. If not, return to the inference state change to perform the parameter update step. If converged, store the experience information, and store the experience information in the database together with the other hand to determine whether the game result meets the expectation. The multi-agent service set shares the experience information set in the database component with all agents in the multi-agent service set to implement parallel execution of the experience information set strategy; The single-agent service set and the multi-agent service set process the experience information to obtain game information, and then encapsulate and store it in the database component.

5. According to claim 1, a multi-level deduction interactive data processing method and system based on intelligent game, characterized in that: The multi-level deduction component obtains observation information by observing the intelligent agent service component; The observation information includes campaign-level observation information and tactical-level observation information.

6. According to claim 1, a multi-level deduction interactive data processing method and system based on intelligent game, characterized in that: The multi-level deduction process of the multi-level deduction component is as follows: In the multi-level deduction process 1, the upper-level deduction action set generates a campaign-level decision through a pulse neural network based on a monotone value function decomposition algorithm, and obtains an upper-level deduction action strategy and sends it to the intermediate deduction action set; In the second multi-level deduction process, the intermediate deduction action level generates tactical-level decisions through a pulse neural network based on a monotone value function decomposition algorithm, and obtains an intermediate deduction action strategy which is sent to a lower-level deduction action set; In multi-level deduction process three, the lower-level deduction action level generates tactical-level decisions through a pulse neural network based on a proximal strategy optimization algorithm.

7. According to claim 1, a multi-level deduction interactive data processing method and system based on intelligent game, characterized in that: The database component receives one third-party environment at a time when receiving the third-party environment access component information; The experience information set includes a training experience information set and a testing experience information set.

8. The experience information set according to claim 7 is a training experience information set and a test experience information set, characterized in that: The training information set and the test experience information set generation process are as follows: When the database component currently uses the third-party environment for the first time, the intelligent agent service component receives the experience information set, which is a test experience information set. The intelligent agent service component processes the test experience information set, encapsulates it, and sends it to the database component to obtain a training experience information set. When the database component continues to use the third-party environment, the intelligent agent service component obtains the training experience information set from the database component, processes and encapsulates it to obtain a new training experience information set and sends it to the database component.

9. The method and system for multi-level deduction interactive data processing based on intelligent game according to claim 1, characterized in that: It also includes a viewing component that can record and store the multi-level deduction process and the experience information, and realize playback and double-speed playback according to the user-developed component.

10. The method and system for processing multi-level deduction interactive data based on intelligent game according to claim 1, characterized in that: The system background component also controls user access to the control unit, generates operation and maintenance status information, and maintains the smooth operation of the system.

Images