Multi-agent thinking link construction method and system based on reinforcement learning

Abstract

The invention discloses a multi-agent thinking link construction method and system based on reinforcement learning, and the method comprises the steps: 1, initializing agents and constructing a thinking link, and employing a multi-agent reinforcement learning framework; 2, the monitoring agent collects confrontation party information in the confrontation scene and feeds back the confrontation party information to the decision-making agent; generating a confrontation target position by the decision-making agent, analyzing a confrontation environment and the state of each agent by using a thinking chain, and generating an action plan; 3, generating an optimal confrontation path and opportunity by the confrontation agent, and executing a confrontation task; and the intelligent agent provides logistics support. According to the method, the chain thinking is introduced into the multi-agent system, and the dynamic information interaction between the agents is utilized to form an effective link, so that the dynamic adaptability and the real-time performance of the link can be effectively improved, the resource consumption of link construction can be reduced, and the decision-making efficiency and the confrontation effect can be improved in a complex and dynamic confrontation environment.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a method and system for constructing a multi-agent thinking link based on reinforcement learning. Background Art

[0002] Traditional command and control systems usually rely on people to process information and make decisions, which requires a lot of manpower and time to deal with complex confrontation environments. In the existing technology, the combination of multi-agent systems and reinforcement learning algorithms can enable agents to collaborate with each other. However, there are often serious information sharing problems between agents, and they cannot flexibly adjust strategies according to task requirements and environmental changes, which makes it difficult to carry out collaborative tasks effectively. First, there is insufficient information sharing. Due to communication bandwidth limitations, security considerations or design defects, agents are often unable to fully exchange necessary state information and intentions, which greatly reduces the effectiveness of collaboration. Secondly, it is difficult to adjust strategies. Most reinforcement learning models are based on fixed scene training and perform poorly when encountering unexpected situations. They cannot flexibly adjust strategies according to task requirements and environmental changes. Third, the coordination mechanism is imperfect, especially in the collaboration between heterogeneous agents. There is a lack of effective solutions, which makes it difficult for agents to collaborate seamlessly and maximize overall benefits. Summary of the invention

[0003] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method and system for constructing a multi-agent thinking link based on reinforcement learning in response to the deficiencies in the prior art.

[0004] In order to solve the above technical problems, in the first aspect, a method for constructing a multi-agent thinking link based on reinforcement learning is disclosed, comprising the following steps:

[0005] Step 1: Initialize the intelligent agent and build a thinking chain. The intelligent agent includes a monitoring agent, a decision-making agent, a confrontation agent, and a guarantee agent. A multi-agent reinforcement learning framework is used. The thinking chain construction enables each intelligent agent to have a coherent reasoning ability based on the current state and enhance the global link coordination.

[0006] Step 2: Monitoring information collection and decision making. The monitoring agent moves in the confrontation scene and collects information about the opponent, and feeds the opponent information back to the decision agent. The decision agent generates the confrontation target position based on the opponent information, uses the thinking chain to analyze the confrontation environment and the status of each agent, and generates an action plan.

[0007] Step 3: The confrontation agent executes and the guarantee agent supports. The confrontation agent generates the optimal confrontation path and timing according to the confrontation target position and executes the confrontation task; the guarantee agent provides logistical support for the monitoring agent and the confrontation agent.

[0008] Furthermore, it also includes step 4, reinforcement learning update and iteration cycle, which updates and optimizes the strategy of each intelligent agent based on the performance feedback of each intelligent agent in the process of building the thinking link, so as to improve the efficiency and response speed of the link; after each confrontation task is completed, it returns to the intelligent agent initialization stage to prepare for the next round of link construction to adapt to the dynamic confrontation environment.

[0009] Furthermore, the agent initialization in step 1 includes initializing the initial position, observation range and action space of each agent. The four types of agents are respectively monitoring (A r ), Decision-making (A d ), Confrontation (A s ) and protection (A b ), initialize the state vector s of each agent i =[p i ,v i ,o i ], where p i is the position, v i is the speed, o i is the observation space; define the strategy π of each agent i and action space a i , where i takes values ​​of r, d, s and b.

[0010] Furthermore, the thought chain construction in step 1 includes each agent decomposing the current state into small-step reasoning through the thought chain method, predicting the possible actions to be taken next, and sharing them with other agents; specifically, each agent in state s i Next, a set of reasoning steps [1,2,…,k,…,K] is generated by thinking chain, K ≥ 1. For each reasoning step k, agent A i The state transition is:

[0011]

[0012] Where f is the state transition function, θ i are the parameters of the agent; the state sequence generated by multi-step reasoning Share with other intelligent agents to build a holistic chain thinking.

[0013] Furthermore, the monitoring information collection in step 2 includes:

[0014] Monitoring Agent A r In the reinforcement learning policy π r (a|s) moves and collects information about the adversary, monitoring agent A r The action selection is done by maximizing the reward function R r accomplish:

[0015] a r = argmax a E[R r (s,a)]

[0016] Among them, s represents the state of the agent, and a represents the actions that the agent can perform in the current state;

[0017] The state update equation of the monitoring agent is:

[0018] s r′ =s r +Δt·v r

[0019] Where Δt is the time step, v r To monitor agent A r The current speed. This equation and the state transfer function f in step 1-2 are state change models at different levels. This equation can be used to describe simple physical actions, and the state transfer function f can be used for more complex and comprehensive state transfer mechanisms.

[0020] Furthermore, the decision making in step 2 includes: decision agent A d Receive the adversary information from the monitoring agent, and based on the state s of the adversary information r and decision agent A d The status of itself d Perform reasoning; decision-making agent A d Using deep reinforcement learning, through the policy π d Determine the adversarial target τ s :

[0021] τ s = argmax τ Q d (s d ,τ;θ d )

[0022] Among them, Q d A is the decision agent d The action value function, θ d is a deep reinforcement learning model parameter, representing the function used to define Q d The parameter of the function, τ, represents the potential adversarial target or action plan; making the decision agent A d The action value function with the largest τ is the adversarial target τ s ;

[0023] Decision-making Agent A d Reasoning about the situation in the confrontation scenario through thought chain method to generate multi-step predictive action plans

[0024] Furthermore, the antagonistic agent in step 3 includes antagonistic agent A s According to the target information τ s Perform adversarial tasks using deep reinforcement learning policy π s Generate the optimal path a s To maximize the confrontation:

[0025]

[0026] in, is the adversarial agent A s The Q function, s s To fight against agent A s Current state, θ s is the parameter of the deep reinforcement learning model, and a represents the adversarial agent A s Actions or paths that can be taken; making the adversary agent A s The a with the largest Q function value is the optimal path a s .

[0027] Furthermore, in step 3, the support of the agent includes: b Provide equipment and logistical support according to the needs of other agents, strategy π b Optimize support and logistics resources b distribution, ensuring agent A b The action selection is based on the maximum reward function R b accomplish:

[0028] a b = argmax a R b (s b ,a)

[0029] Among them, s b Represents the protection agent A b The state vector, the maximum reward function R b Used to optimize support decisions, a represents the security agent A b Actions that can be taken.

[0030] Furthermore, the reinforcement learning update and iteration cycle in step 4 includes:

[0031] Step 4-1: Reinforcement learning update: Use the multi-agent reinforcement learning framework to update the agent strategy, combining experience replay and target network; each agent updates the strategy parameter θ after completing the round i :

[0032]

[0033] Among them, α is the learning rate, γ is the discount factor, t represents each time point in the time series, and each t corresponds to a specific state s t 、Action a t and its corresponding immediate reward R i (s t ,a t ), the state transition information provided by the environment is automatically recorded.

[0034] Step 4-2, iterative loop, after completing each task, each agent is reinitialized and proceeds to the next loop.

[0035] In the second aspect, a multi-agent thinking link construction system based on reinforcement learning is disclosed, including: an agent initialization module, a thinking chain module, a monitoring information collection module, a decision-making designation module, an execution module, a support module and an update module.

[0036] The agent initialization module is used to initialize the state vectors of the monitoring agent, decision-making agent, confrontation agent and guarantee agent, define the strategy and action space of each agent, and the agent uses a multi-agent reinforcement learning framework;

[0037] The thought chain module is used for each intelligent agent to decompose the current state into small-step reasoning through the thought chain method, predict the possible actions to be taken next, and share them with other intelligent agents;

[0038] The monitoring information collection module is used to monitor the movement of the agent in the confrontation scenario and collect information about the opponent, and feed the opponent information back to the decision-making agent;

[0039] The decision-making designation module is used for the decision-making agent to generate the confrontation target position based on the confrontation party information, use the thinking chain to analyze the confrontation environment and the status of each agent, and generate an action plan;

[0040] The execution module is used for the confrontation agent to generate the optimal confrontation path and timing according to the confrontation target position and execute the confrontation task;

[0041] The support module is used to ensure that the intelligent agent provides logistical support for the monitoring intelligent agent and the confrontation intelligent agent;

[0042] The update module is used to update and optimize the strategy of each intelligent agent based on the performance feedback of each intelligent agent during the thinking link construction process; after each confrontation task is completed, it returns to the intelligent agent initialization stage to prepare for the next round of link construction.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention significantly improves the collaborative efficiency and autonomous decision-making ability of monitoring, decision-making, confrontation and protection agents in confrontation scenarios through the optimization of multi-agent systems, the application of reinforcement learning technology and thought chain, realizes efficient sharing and communication of information, and enhances adaptability to complex dynamic environments. Compared with the existing technology, the present invention has shown obvious advantages in both experimental verification and practical applications, greatly improving confrontation effectiveness and task execution accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0046] Figure 1 A schematic diagram of a multi-agent thinking link construction method based on reinforcement learning provided in the embodiment of the present application Figure 1 .

[0047] Figure 2 A schematic diagram of a multi-agent thinking link construction method based on reinforcement learning provided in the embodiment of the present application Figure 2 .

[0048] Figure 3 A framework diagram of a multi-agent thinking link construction system based on reinforcement learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0050] This application provides a method and system for building a multi-agent thinking link based on reinforcement learning. By integrating a multi-agent system, reinforcement learning algorithms, multi-source perception technology and an efficient communication network, it realizes efficient coordination and autonomous optimization of monitoring, decision-making, confrontation and guarantee agents. Specifically, the monitoring agent uses advanced sensor technology to collect information about the opponent, the decision-making agent integrates multi-source data to carry out scientific combat deployment, the confrontation agent eliminates the target through precise positioning and rapid confrontation, and the guarantee agent provides necessary logistical support.

[0051] The present invention uses hierarchical reinforcement learning and multi-agent collaborative technology, introduces chain thinking into the multi-agent system, and combines it with a reinforcement learning framework to optimize the real-time link connection between monitoring and confrontation. Each agent can dynamically adjust its strategy, improve task execution efficiency and decision-making accuracy, and enhance its adaptability to complex dynamic environments, thereby comprehensively improving the confrontation effectiveness in confrontation scenarios. This application not only optimizes information sharing and communication, but also realizes intelligence and autonomy. It can be applied to intelligent traffic management systems, disaster response and rescue coordination scenarios.

[0052] The first embodiment of the present application discloses a method for constructing a multi-agent thinking link based on reinforcement learning, such as Figure 1 and Figure 2 As shown, the following steps are included:

[0053] Step 1: Initialize the intelligent agent and build a thinking chain. The intelligent agents include monitoring agents, decision-making agents, confrontation agents, and assurance agents, using a multi-agent reinforcement learning framework. Initialize the four types of intelligent agents: monitoring, decision-making, confrontation, and assurance, and set their initial positions, observation ranges, and action spaces. Each intelligent agent decomposes the current state into small-step reasoning through the construction of a thinking chain, predicts the possible actions to be taken in the next step, and shares them with other intelligent agents. The core of the thinking chain construction is to enable each intelligent agent to have the ability of coherent reasoning based on its current state and enhance the coordination of the global link.

[0054] In the specific implementation process, the multi-agent reinforcement learning framework can use frameworks such as Ray RLLib and PettingZoo.

[0055] Step 1 includes:

[0056] Step 1-1, agent initialization, assuming that the four types of agents are monitoring (A r ), Decision-making (A d ), Confrontation (A s ) and protection (A b ). Initialize the state vector s of each agent i =[p i ,v i ,o i ], where p i is the position, v i is the speed, o i is the observation space. Define the strategy π of each agent i and action space a i , where i takes values ​​of r, d, s and b.

[0057] Step 1-2: Thinking chain construction, each agent is in state s iNext, a set of reasoning steps [1,2,…,k,…,K] is generated by thinking chain, where K≥1. For each reasoning step k, agent A i The state transition is:

[0058]

[0059] Where f is the state transition function, θ i are the parameters of the agent. The state sequence generated by multi-step reasoning Share with other intelligent agents to build a holistic chain thinking.

[0060] In the specific implementation process, examples of the application of the thinking chain method in the intelligent traffic management system are: 1. The monitoring agent is responsible for monitoring traffic flow and monitoring traffic accidents; 2. The decision-making agent specifies the optimal traffic management strategy based on the information provided by the monitoring agent; 3. The confrontation agent responds to emergencies; 4. The security agent ensures the normal operation of the system and provides necessary support and services.

[0061] Step 2: Monitoring information collection and decision making. The monitoring agent moves in the confrontation scene and collects information about the opponent through reinforcement learning strategies. The opponent information includes information such as the opponent's location and equipment. The opponent information is fed back to the decision-making agent; the decision-making agent uses the deep reinforcement learning algorithm to analyze the intelligence provided by the monitoring agent and generate the target location and timing of the opponent agent. The agent uses the thinking chain to analyze the confrontation environment and the status of each agent and generate an action plan.

[0062] Step 2 includes:

[0063] Step 2-1: Monitoring information collection, monitoring agent A r Move and collect enemy information under the reinforcement learning strategy πr(a|s). Its action selection is achieved by maximizing the reward function R r accomplish:

[0064] a r = argmax a E[R r (s,a)]

[0065] Among them, s represents the state of the agent, and a represents the actions that the agent can perform in the current state;

[0066] The state update equation of the monitoring agent is:

[0067] s r′ =s r +Δt·v r

[0068] Where Δt is the time step, vr To monitor the current speed of the intelligent agent Ar.

[0069] Step 2-2: Decision making, decision agent A d Receive the adversary information from the monitoring agent, and based on the state s of the adversary information r and decision agent A d The status of itself d Perform reasoning. Decision-making agent A d Using deep reinforcement learning, through the policy π d Determine the adversarial target τ s :

[0070] τ s = argmax τ Q d (s d ,τ;θ d )

[0071] in, A is the decision agent d The action value function, θ d is a deep reinforcement learning model parameter, representing the The parameter of the function, τ, represents the potential adversarial target or action plan; making the decision agent A d The action value function with the largest τ is the adversarial target τ s .

[0072] Decision-making Agent A d Reasoning about the situation in the confrontation scenario through thought chaining to generate multi-step predictive action plans

[0073] Step 3: The adversarial agent executes and the guarantee agent supports. The adversarial agent uses a deep reinforcement learning algorithm to generate the optimal adversarial path and timing based on the adversarial target information provided by the decision-making agent, and executes the adversarial task. The guarantee agent provides necessary logistical support to the monitoring and adversarial agents, including equipment replenishment, maintenance and other tasks, to ensure the continuous operation of the link.

[0074] Step 3 includes:

[0075] Step 3-1: Execution of adversarial agent, adversarial agent A s According to the target information τ s Perform adversarial tasks. Use deep reinforcement learning policy π s Generate the optimal path a s To maximize the confrontation:

[0076]

[0077] in is the adversarial agent A s The Q function, s s To fight against agent A s Current state, θ s is the parameter of the deep reinforcement learning model, and a represents the adversarial agent A s Actions or paths that can be taken; making the adversary agent A s The a with the largest Q function value is the optimal path a s .

[0078] Step 3-2: Ensure agent support, ensure agent A b Provide equipment and logistical support according to the needs of other agents. Its strategy π b Optimized support and logistics resources b distribution, ensuring agent A b The action selection is based on the maximum reward function R b accomplish:

[0079] a b = argmax a R b (s b ,a)

[0080] Among them, s b Represents the protection agent A b The state vector, the maximum reward function R b Used to optimize support decisions, a represents the security agent A b Actions that can be taken.

[0081] The present embodiment provides a method for constructing a multi-agent thinking link based on reinforcement learning, which also includes step 4, reinforcement learning update and iteration cycle. Based on the performance feedback of each agent in the process of building the thinking link, the strategy of each agent is updated and optimized to continuously improve the efficiency and response speed of the link. After each confrontation task is completed, it returns to the agent initialization stage to prepare for the next round of link construction to adapt to the dynamic confrontation environment.

[0082] Step 4 includes:

[0083] Step 4-1: Reinforcement learning update: Use the multi-agent reinforcement learning framework to update the agent strategy, combining experience replay and target network. Each agent updates the strategy parameter θ after completing the round. i :

[0084]

[0085] Among them, α is the learning rate, γ is the discount factor, t represents each time point in the time series, and each t corresponds to a specific state st 、Action a t and its corresponding immediate reward R i (s t ,a t ), the state transition information provided by the environment is automatically recorded.

[0086] Step 4-2, iterative loop, after completing each task, each agent is reinitialized and proceeds to the next loop.

[0087] The second embodiment of the present application discloses a multi-agent thinking link construction system based on reinforcement learning, such as Figure 3 As shown, it includes an agent initialization module, a thinking chain module, a monitoring information collection module, a decision-making module, an execution module, a support module and an update module.

[0088] The agent initialization module is used to initialize the state vectors of the monitoring agent, decision-making agent, confrontation agent and guarantee agent, define the strategy and action space of each agent, and the agent uses a multi-agent reinforcement learning framework;

[0089] The thought chain module is used for each intelligent agent to decompose the current state into small-step reasoning through the thought chain method, predict the possible actions to be taken next, and share them with other intelligent agents;

[0090] The monitoring information collection module is used to monitor the movement of the agent in the confrontation scenario and collect information about the opponent, and feed the opponent information back to the decision-making agent;

[0091] The decision-making designation module is used for the decision-making agent to generate the confrontation target position based on the confrontation party information, use the thinking chain to analyze the confrontation environment and the status of each agent, and generate an action plan;

[0092] The execution module is used for the confrontation agent to generate the optimal confrontation path and timing according to the confrontation target position and execute the confrontation task;

[0093] The support module is used to ensure that the intelligent agent provides logistical support for the monitoring intelligent agent and the confrontation intelligent agent;

[0094] The update module is used to update and optimize the strategy of each intelligent agent based on the performance feedback of each intelligent agent during the thinking link construction process; after each confrontation task is completed, it returns to the intelligent agent initialization stage to prepare for the next round of link construction.

[0095] In actual deployment, the problems faced by the present invention include complex adversarial environments, stability and security of communication networks, real-time and reliability of multi-agent collaborative operations, and high computing resource requirements. To overcome these difficulties, the following measures can be taken: improve the perception and decision-making algorithms of the agents to enhance their ability to adapt to complex environments; optimize communication protocols and encryption technologies to ensure the stability and security of communications; use distributed computing and edge computing technologies to improve the real-time and reliability of the system; balance performance and cost through algorithm optimization and hardware upgrades to improve the robustness and efficiency of the overall system.

[0096] In summary, by optimizing the collaborative decision-making and task allocation of the multi-agent system, the efficiency and autonomous decision-making ability of the four types of agents are significantly improved, efficient information sharing and communication are achieved, and the adaptability to complex dynamic environments is enhanced, thereby greatly improving the confrontation effectiveness and task execution accuracy. The training results were experimentally analyzed. Taking the intelligent traffic management system as an example, the urban traffic management system was simulated. By introducing hierarchical reinforcement learning and chain thinking modules, compared with traditional technologies, the average commuting time was shortened by about 30%, the accident response time was reduced by 40%, the system resource utilization rate was increased by 25%, and the agent collaboration efficiency was increased by 50%, proving that the system designed by the present invention has good adaptability, and the method and model proposed by the present invention have certain applicability. A multi-agent thinking link construction method and system based on reinforcement learning proposed in this application overcome the defects of the prior art and improve the collaboration ability and decision-making efficiency of the multi-agent system in a complex confrontation environment. Specifically, the present invention enhances information sharing between agents, the developed adaptive reinforcement learning algorithm can flexibly adjust strategies, and the designed coordination framework suitable for heterogeneous agents ensures that they can collaborate efficiently. At the same time, the excellent performance of agents in real-time and reliability can be guaranteed by adopting a distributed computing architecture and a high-performance hardware platform. These improvements are crucial to improving the operational efficiency and reliability of multi-agent systems, and lay a solid foundation for the construction of future intelligent command and control systems.

[0097] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, the invention content of the method for building a multi-agent thinking link based on reinforcement learning provided by the present invention and some or all of the steps in each embodiment can be run. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0098] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention are essentially or partly contributed to the prior art can be embodied in the form of a computer program, i.e., a software product, which can be stored in a storage medium and includes several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, a MUU or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0099] The present invention provides a method and system for constructing a multi-agent thinking link based on reinforcement learning. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A method for constructing a multi-agent thinking link based on reinforcement learning, characterized in that: The following steps are involved: Step 1: Initialize the intelligent agent and build a thinking chain. The intelligent agent includes a monitoring agent, a decision-making agent, a confrontation agent, and a guarantee agent. A multi-agent reinforcement learning framework is used. The thinking chain construction enables each intelligent agent to have a coherent reasoning ability based on its current state and enhance the global link coordination. Step 2: Monitoring information collection and decision making. The monitoring agent moves in the confrontation scenario and collects information about the opponent through a reinforcement learning strategy, and feeds the opponent information back to the decision agent. The decision agent generates the target position for confrontation based on the opponent information. The agent uses a thinking chain to analyze the confrontation environment and the status of each agent and generates an action plan. Step 3: The adversarial agent executes and the guarantee agent supports. The adversarial agent uses a deep reinforcement learning algorithm to generate the optimal adversarial path and timing based on the adversarial target information and executes the adversarial task. The guarantee agent is responsible for providing logistical support for the monitoring agent and the adversarial agent.

2. According to a method for constructing a multi-agent thinking link based on reinforcement learning according to claim 1, it is characterized by: It also includes step 4, reinforcement learning update and iteration cycle, which updates and optimizes the strategy of each intelligent agent based on the performance feedback of each intelligent agent during the thinking link construction process; after each confrontation task is completed, it returns to the intelligent agent initialization stage to prepare for the next round of link construction.

3. According to claim 2, a method for constructing a multi-agent thinking link based on reinforcement learning is characterized in that: The agent initialization in step 1 includes initializing the initial position, observation range and action space of each agent. Suppose the four types of agents are monitoring (A r ), Decision-making (A d ), Confrontation (A s ) and protection (A b ), initialize the state vector s of each agent i =[p i ,v i ,o i ], where p i is the position, v i is the speed, o i is the observation space; define the strategy π of each agent i and action space a i , where i takes values ​​of r, d, s and b.

4. According to claim 3, a method for constructing a multi-agent thinking link based on reinforcement learning is characterized in that: The thought chain construction in step 1 includes each agent decomposing the current state into small-step reasoning through the thought chain method, predicting the possible actions to be taken next, and sharing them with other agents; specifically, each agent in state s i Next, a set of reasoning steps [1,2,…,k,…,K] is generated by thinking chain, K ≥ 1. For each reasoning step k, agent A i The state transition is: Where f is the state transition function, θ i are the parameters of the agent; the state sequence generated by multi-step reasoning Share with other intelligent agents to build a holistic chain thinking.

5. According to claim 4, a method for constructing a multi-agent thinking link based on reinforcement learning is characterized in that: The monitoring information collection in step 2 includes: Monitoring Agent A r In the reinforcement learning policy π r (a|s) moves and collects information about the adversary, monitoring agent A r The action selection is done by maximizing the reward function R r accomplish: a r =arg max a E[R r (s,a)] Among them, s represents the state of the agent, and a represents the action that the agent can perform in the current state; The state update equation of the monitoring agent is: s r′ =s r +Δt·v r Where Δt is the time step, v r To monitor agent A r The current speed.

6. A method for constructing a multi-agent thinking link based on reinforcement learning according to claim 5, characterized in that: The decision making in step 2 includes: Decision Agent A d Receive the adversary information from the monitoring agent, and based on the state s of the adversary information r and decision agent A d The status of itself d Perform reasoning; decision-making agent A d Using deep reinforcement learning, through the policy π d Determine the adversarial target τ s : in, A is the decision agent d The action value function, θ d is a deep reinforcement learning model parameter, representing the The parameter of the function, τ, represents the potential adversarial target or action plan; making the decision agent A d The action value function with the largest τ is the adversarial target τ s ; Decision-making Agent A d Reasoning about the situation in the confrontation scenario through thought chain method to generate multi-step predictive action plans 7. A method for constructing a multi-agent thinking link based on reinforcement learning according to claim 6, characterized in that: The adversarial agent execution in step 3 includes adversarial agent A s According to the target information τ s Perform adversarial tasks using deep reinforcement learning policy π s Generate the optimal path a s To maximize the confrontation: in, is the adversarial agent A s The Q function, s s To fight against agent A s Current state, θ s is the parameter of the deep reinforcement learning model, and a represents the adversarial agent A s Actions or paths that can be taken; making the adversary agent A s The a with the largest Q function value is the optimal path a s .

8. A method for constructing a multi-agent thinking link based on reinforcement learning according to claim 7, characterized in that: In step 3, the support of the security agent includes: Security agent A b Provide equipment and logistical support according to the needs of other agents, strategy π b Optimize support and logistics resources b distribution, ensuring agent A b The action selection is based on the maximum reward function R b accomplish: a b =arg max a R b (s b ,a) Among them, s b Represents the protection agent A b The state vector, maximum reward function R b Used to optimize support decisions, a represents the security agent A b Actions that can be taken.

9. A method for constructing a multi-agent thinking link based on reinforcement learning according to claim 8, characterized in that: The reinforcement learning update and iteration cycle in step 4 includes: Step 4-1: Reinforcement learning update: Use the multi-agent reinforcement learning framework to update the agent strategy, combining experience replay and target network; each agent updates the strategy parameter θ after completing the round i : Among them, α is the learning rate, γ is the discount factor, t represents each time point in the time series, and each t corresponds to a state s t 、Action a t and its corresponding immediate reward R i (s t ,a t ), the state transition information provided by the environment is automatically recorded; Step 4-2, iterative loop, after completing each task, each agent is reinitialized and proceeds to the next loop.

10. A multi-agent thinking link construction system based on reinforcement learning, characterized by: It includes the agent initialization module, the thinking chain module, the monitoring information collection module, the decision-making module, the execution module, the support module and the update module. The agent initialization module is used to initialize the state vectors of the monitoring agent, decision-making agent, confrontation agent and guarantee agent, define the strategy and action space of each agent, and the agent uses a multi-agent reinforcement learning framework; The thought chain module is used for each intelligent agent to decompose the current state into small-step reasoning through the thought chain method, predict the possible actions to be taken next, and share them with other intelligent agents; The monitoring information collection module is used to monitor the movement of the agent in the confrontation scenario and collect information about the opponent, and feed the opponent information back to the decision-making agent; The decision-making designation module is used for the decision-making agent to generate the confrontation target position based on the confrontation party information, use the thinking chain to analyze the confrontation environment and the status of each agent, and generate an action plan; The execution module is used for the confrontation agent to generate the optimal confrontation path and timing according to the confrontation target position and execute the confrontation task; The support module is used to ensure that the intelligent agent provides logistical support for the monitoring intelligent agent and the confrontation intelligent agent; The updating module is used to update and optimize the strategies of each intelligent agent based on the performance feedback of each intelligent agent during the process of building the thinking link; After each confrontation task is completed, it returns to the agent initialization stage to prepare for the next round of link construction.

Images