A multi-intelligent agent cooperative interaction method and system

Abstract

The application discloses a multi-intelligent agent cooperative interaction method and system, relates to the technical field of multi-agent cooperative interaction, and is characterized in that each intelligent agent observes a local state and acquires local state information of a neighbor intelligent agent through local network communication; each intelligent agent selects an action by using a reinforcement learning algorithm according to the local state and the neighbor local state; the intelligent agent broadcasts the selected action information to the neighbor intelligent agent through the local network communication, and the neighbor intelligent agent adjusts a Q value function of the neighbor intelligent agent according to the received action information. The application realizes efficient, robust and scalable cooperative decision-making through distributed reinforcement learning, local communication and a consistency protocol. The application improves system robustness, reduces communication overhead, enhances dynamic environment adaptability, guarantees global consistency, improves task completion efficiency, supports heterogeneous agents, reduces calculation complexity, enhances scalability, improves learning efficiency and supports multi-task cooperation.

Description

Technical Field

[0001] The present invention relates to the field of multi-agent collaborative interaction technology, and in particular to a multi-intelligent agent collaborative interaction method and system. Background Art

[0002] Multi-agent collaborative interaction systems are a rapidly developing research area in artificial intelligence. These systems consist of multiple autonomous agents, each of which can independently perceive the environment and take actions to achieve a given goal.

[0003] Multi-Agent Systems (MAS) are widely used in areas such as collaborative robotics, autonomous driving, intelligent transportation, and smart grids. These systems accomplish complex tasks through collaboration and communication between multiple agents. Common collaborative approaches include centralized control and distributed control.

[0004] Centralized control is a central controller that coordinates the behavior of all intelligent agents, which has the defects of single point of failure and communication bottleneck;

[0005] Distributed control is a system where each intelligent agent makes independent decisions and collaborates through local communication. This has the drawback of being difficult to ensure global consistency.

[0006] Existing systems have poor adaptability in dynamic environments and struggle to respond quickly to environmental changes. As the number of intelligent agents increases, computational complexity grows exponentially, impacting real-time performance. Summary of the Invention

[0007] In order to solve the above technical problems, the present invention provides a multi-agent collaborative interaction method and system. The following technical solutions are adopted:

[0008] A multi-agent collaborative interaction method comprises the following steps:

[0009] Step 1: Define a set of intelligent agents and initialize the local state and action space of each intelligent agent;

[0010] Step 2: Each intelligent agent observes its local state and obtains the local state information of its neighboring intelligent agents through local network communication.

[0011] Step 3: Each intelligent agent uses a reinforcement learning algorithm to select an action based on its local state and the local states of its neighbors.

[0012] Step 4: The intelligent agent broadcasts the selected action information to neighboring intelligent agents through local network communication. The neighboring intelligent agents adjust their own Q-value functions according to the received action information.

[0013] Step 5: Introduce a consensus protocol to ensure that local decisions among intelligent agents tend to be consistent;

[0014] In step 6, each intelligent agent periodically updates its local state and neighbor local states, and adjusts the Q-value function according to environmental changes.

[0015] By adopting the above technical solutions, this multi-agent collaborative interaction method can effectively solve the collaboration and decision-making problems in multi-agent systems by combining distributed reinforcement learning and consensus protocols.

[0016] The system does not rely on a central controller. Each intelligent agent makes independent decisions and collaborates through local communication. A single intelligent agent failure does not cause the entire system to crash. Even in the event of communication interruptions or partial agent failure, the system continues to operate through local communication and distributed decision-making. Through local network communication, intelligent agents exchange information only with their neighbors, avoiding the high overhead of global communication. This significantly reduces communication bandwidth requirements, making it suitable for resource-constrained scenarios (such as drone fleets and IoT devices). The system is highly scalable, and communication overhead does not increase exponentially with the number of intelligent agents.

[0017] The intelligent agent can quickly respond to environmental changes by regularly updating its local state and adjusting its Q-value function. In dynamic environments (such as moving obstacles or changing target positions), the system can adjust its strategy in real time.

[0018] Adaptive learning rate and discount factor further improve the adaptability of the system in uncertain environments.

[0019] By introducing a consensus protocol, local decisions among intelligent agents tend to be consistent, avoiding conflicts and waste of resources.

[0020] For example, in a multi-robot collaborative handling task, all robots can move in coordination to avoid collisions with each other.

[0021] In a smart grid, distributed energy units can coordinate and adjust output power to ensure grid stability.

[0022] Intelligent agents optimize decisions through reinforcement learning and are able to complete tasks efficiently.

[0023] For example, in target search tasks, intelligent agents can quickly find the optimal path. In resource allocation tasks, intelligent agents can allocate resources fairly and efficiently.

[0024] This method does not rely on the homogeneity of intelligent agents and can support the collaborative work of agents of different types and capabilities. For example, in a collaborative task between a drone and a ground robot, different types of agents can work together according to their capabilities.

[0025] In an intelligent transportation system, vehicles, pedestrians, and traffic lights work together to optimize traffic flow.

[0026] Through distributed decision-making and local communication, each intelligent agent only needs to process local information, reducing computational complexity. Computational resource requirements are dispersed across the intelligent agents, avoiding centralized computing bottlenecks.

[0027] The system can run in real time and is suitable for scenarios with high requirements for response speed (such as autonomous driving and robot collaboration).

[0028] The method is applicable to small- to large-scale multi-agent systems and can be flexibly expanded. For example, in an intelligent warehousing system, as the number of robots increases, the system can automatically adjust the collaboration strategy.

[0029] Intelligent agents can accelerate the learning process by sharing local state and action information.

[0030] The method can process multiple tasks simultaneously and achieve collaborative optimization between tasks.

[0031] For example, in smart factories, robots can simultaneously perform handling, assembly, and inspection tasks, optimizing overall production efficiency.

[0032] This multi-agent collaborative interaction method achieves efficient, robust, and scalable collaborative decision-making through distributed reinforcement learning, local communication, and a consensus protocol. It improves system robustness, reduces communication overhead, enhances adaptability to dynamic environments, ensures global consistency, improves task completion efficiency, supports heterogeneous agents, reduces computational complexity, enhances scalability, improves learning efficiency, and supports multi-task collaboration. These benefits make this method promising for broad application in areas such as robotic collaboration, intelligent transportation, smart grids, and drone formations.

[0033] Optionally, in step 6, the Q-value function is adjusted to introduce an adaptive learning rate and a discount factor to improve the adaptability of the collaborative interaction of multiple intelligent agents in a dynamic environment.

[0034] By adopting the above technical solutions, the adaptive learning rate can dynamically adjust the learning rate according to changes in the current environment, enabling the intelligent agent to adapt to the new environment more quickly and improve learning efficiency.

[0035] Adaptive learning rate can avoid overfitting of intelligent agents during training and improve the generalization ability of the model.

[0036] Adaptive learning rate can improve the robustness of intelligent agents in dynamic environments, enabling them to better cope with challenges brought by environmental changes.

[0037] Balancing short-term and long-term gains: The discount factor can balance the short-term and long-term gains of the intelligent agent in a dynamic environment, enabling it to pursue short-term goals while also considering long-term goals.

[0038] The discount factor can help intelligent agents better evaluate the long-term value of different decisions and thus make higher-quality decisions.

[0039] The discount factor can improve the adaptability of intelligent agents in dynamic environments, enabling them to better cope with the challenges brought by environmental changes.

[0040] Optionally, the Q-value function calculation formula is:

[0041] ;

[0042] in Indicates that the i-th intelligent agent is in the local state and neighbor status Next, select Action Q value;

[0043] is the learning rate, ranging from ; is the immediate reward, indicating that the intelligent agent i is performing an action Direct rewards received after is the discount factor, ranging from ; Indicates the next local state of the intelligent agent i and neighbor local states Select Action The maximum Q value of is the local state of the intelligent agent i at the next moment; is the local state of the neighbors of the i-th intelligent agent at the next moment.

[0044] Optionally, in step 5, the consistency protocol formula is:

[0045] ;

[0046] in Represents the local state of intelligent agent i; The local state of the neighboring agent j of agent i; represents the neighbor set of intelligent agent i, which is determined by the communication range and task requirements; is a normalization factor used to calculate the average of the local states of agent i and its neighbors.

[0047] Optionally, local state is position, velocity, and sensor data.

[0048] Optionally, the action selection formula in step 3 is:

[0049] ;

[0050] in represents the action selected by intelligent agent i; is the set of actions of intelligent agent i; Indicates that the intelligent agent i is in the local state and neighbor local states Select Action Q value.

[0051] Optionally, actions are movement direction, speed adjustment, and task allocation.

[0052] A multi-intelligent agent collaborative interaction system is used to implement a multi-intelligent agent collaborative interaction method. The multi-intelligent agent collaborative interaction system includes multiple intelligent agents and a communication network module. The intelligent agent includes an intelligent motion body and a collaborative interaction module. The collaborative interaction module is installed on the intelligent motion body and is communicatively connected to the main control chip of the intelligent motion body. The collaborative interaction module includes an interactive wireless communication module, a computing module and a sensor module. The interactive wireless communication module is wirelessly connected to the interactive wireless communication module of the adjacent intelligent agent based on the local network communication formed by the communication network module. The computing module is communicatively connected to the interactive wireless communication module and the sensor module. The sensor module is used to monitor the local state of the intelligent motion body.

[0053] Optionally, the communication network module forms a local network communication within the operating range of multiple intelligent moving bodies based on ZigBee or Lora wireless communication technology, and the interactive wireless communication module is wirelessly connected to the interactive wireless communication module of the adjacent intelligent agent body based on ZigBee or Lora wireless communication.

[0054] Optionally, the computing module includes a cache and a processor, the cache is communicatively connected to the interactive wireless communication module, the processor is communicatively connected to the cache, and the sensor module includes a camera, a lidar and an inertial measurement unit, and the camera, lidar and inertial measurement unit are respectively communicatively connected to the cache.

[0055] By adopting the above technical solution, the intelligent moving agents, such as robots and drones, can interact with each other in real time, improving collaborative efficiency. The sensor module can acquire environmental information surrounding the intelligent agents, providing a basis for their decision-making. The sensor module can enhance the intelligent agents' perception capabilities, enabling them to better adapt to environmental changes.

[0056] ZigBee or Lora wireless communication technology has the characteristics of low power consumption and can extend the working time of the intelligent agent.

[0057] In summary, the present invention includes at least one of the following beneficial technical effects:

[0058] The multi-agent collaborative interaction method can effectively solve the collaboration and decision-making problems in multi-agent systems by combining distributed reinforcement learning and consensus protocols.

[0059] Independent of a central controller, each intelligent agent makes independent decisions and collaborates through local communication. A single intelligent agent failure does not cause the entire system to crash. Even in the event of communication interruptions or partial agent failure, the system continues to operate through local communication and distributed decision-making. Through local network communication, intelligent agents exchange information only with their neighbors, avoiding the high overhead of global communication. This significantly reduces communication bandwidth requirements, making it suitable for resource-constrained scenarios (such as drone fleets and IoT devices). The system is highly scalable, and communication overhead does not increase exponentially with the number of intelligent agents.

[0060] The intelligent agent can quickly respond to environmental changes by regularly updating its local state and adjusting its Q-value function. In dynamic environments (such as moving obstacles or changing target positions), the system can adjust its strategy in real time.

[0061] Adaptive learning rate and discount factor further improve the adaptability of the system in uncertain environments.

[0062] By introducing a consensus protocol, local decisions among intelligent agents tend to be consistent, avoiding conflicts and waste of resources.

[0063] This multi-agent collaborative interaction method achieves efficient, robust, and scalable collaborative decision-making through distributed reinforcement learning, local communication, and a consensus protocol. This method improves system robustness, reduces communication overhead, enhances adaptability to dynamic environments, ensures global consistency, improves task completion efficiency, supports heterogeneous agents, reduces computational complexity, enhances scalability, improves learning efficiency, and supports multi-task collaboration. These benefits make this method promising for broad application in areas such as robotic collaboration, intelligent transportation, smart grids, and drone formations. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of a multi-intelligent agent collaborative interaction method of the present invention;

[0065] Figure 2 This is a schematic diagram of the electrical component connection principle of a multi-intelligent agent collaborative interaction system of the present invention;

[0066] Figure 3 This is a schematic diagram of the connection principle of electrical components of a collaborative interaction module of a multi-intelligent agent collaborative interaction system of the present invention.

[0067] Explanation of the accompanying drawings: 11. Intelligent motion body; 12. Collaborative interaction module; 121. Interactive wireless communication module; 122. Computing module; 1221. Cache; 1222. Processor; 123. Sensor module; 1231. Camera; 1232. LiDAR; 1233. Inertial measurement unit; 2. Communication network module. DETAILED DESCRIPTION

[0068] The present invention will be further described in detail below with reference to the accompanying drawings.

[0069] The embodiment of the present invention discloses a multi-intelligent agent collaborative interaction method and system.

[0070] Reference Figure 1-Figure 3 ,Example 1, a multi-intelligent agent collaborative interaction method, comprising the following steps:

[0071] Step 1: Define a set of intelligent agents and initialize the local state and action space of each intelligent agent;

[0072] Step 2: Each intelligent agent observes its local state and obtains the local state information of its neighboring intelligent agents through local network communication.

[0073] Step 3: Each intelligent agent uses a reinforcement learning algorithm to select an action based on its local state and the local states of its neighbors.

[0074] Step 4: The intelligent agent broadcasts the selected action information to neighboring intelligent agents through local network communication. The neighboring intelligent agents adjust their own Q-value functions according to the received action information.

[0075] Step 5: Introduce a consensus protocol to ensure that local decisions among intelligent agents tend to be consistent;

[0076] In step 6, each intelligent agent periodically updates its local state and neighbor local states, and adjusts the Q-value function according to environmental changes.

[0077] This multi-agent collaborative interaction method can effectively solve the collaboration and decision-making problems in multi-agent systems by combining distributed reinforcement learning and consensus protocols.

[0078] The system does not rely on a central controller. Each intelligent agent makes independent decisions and collaborates through local communication. A single intelligent agent failure does not cause the entire system to crash. Even in the event of communication interruptions or partial agent failure, the system continues to operate through local communication and distributed decision-making. Through local network communication, intelligent agents exchange information only with their neighbors, avoiding the high overhead of global communication. This significantly reduces communication bandwidth requirements, making it suitable for resource-constrained scenarios (such as drone fleets and IoT devices). The system is highly scalable, and communication overhead does not increase exponentially with the number of intelligent agents.

[0079] The intelligent agent can quickly respond to environmental changes by regularly updating its local state and adjusting its Q-value function. In dynamic environments (such as moving obstacles or changing target positions), the system can adjust its strategy in real time.

[0080] Adaptive learning rate and discount factor further improve the adaptability of the system in uncertain environments.

[0081] By introducing a consensus protocol, local decisions among intelligent agents tend to be consistent, avoiding conflicts and waste of resources.

[0082] For example, in a multi-robot collaborative handling task, all robots can move in coordination to avoid collisions with each other.

[0083] In a smart grid, distributed energy units can coordinate and adjust output power to ensure grid stability.

[0084] Intelligent agents optimize decisions through reinforcement learning and are able to complete tasks efficiently.

[0085] For example, in target search tasks, intelligent agents can quickly find the optimal path. In resource allocation tasks, intelligent agents can allocate resources fairly and efficiently.

[0086] This method does not rely on the homogeneity of intelligent agents and can support the collaborative work of agents of different types and capabilities. For example, in a collaborative task between a drone and a ground robot, different types of agents can work together according to their capabilities.

[0087] In an intelligent transportation system, vehicles, pedestrians, and traffic lights work together to optimize traffic flow.

[0088] Through distributed decision-making and local communication, each intelligent agent only needs to process local information, reducing computational complexity. Computational resource requirements are dispersed across the intelligent agents, avoiding centralized computing bottlenecks.

[0089] The system can run in real time and is suitable for scenarios with high requirements for response speed (such as autonomous driving and robot collaboration).

[0090] The method is applicable to small- to large-scale multi-agent systems and can be flexibly expanded. For example, in an intelligent warehousing system, as the number of robots increases, the system can automatically adjust the collaboration strategy.

[0091] Intelligent agents can accelerate the learning process by sharing local state and action information.

[0092] The method can process multiple tasks simultaneously and achieve collaborative optimization between tasks.

[0093] For example, in smart factories, robots can simultaneously perform handling, assembly, and inspection tasks, optimizing overall production efficiency.

[0094] This multi-agent collaborative interaction method achieves efficient, robust, and scalable collaborative decision-making through distributed reinforcement learning, local communication, and a consensus protocol. It improves system robustness, reduces communication overhead, enhances adaptability to dynamic environments, ensures global consistency, improves task completion efficiency, supports heterogeneous agents, reduces computational complexity, enhances scalability, improves learning efficiency, and supports multi-task collaboration. These benefits make this method promising for broad application in areas such as robotic collaboration, intelligent transportation, smart grids, and drone formations.

[0095] In Example 2, in step 6, the Q-value function is adjusted to introduce an adaptive learning rate and a discount factor to improve the adaptability of the collaborative interaction of multiple intelligent agents in a dynamic environment.

[0096] The adaptive learning rate can dynamically adjust the learning rate according to changes in the current environment, enabling the intelligent agent to adapt to the new environment more quickly and improve learning efficiency.

[0097] Adaptive learning rate can avoid overfitting of intelligent agents during training and improve the generalization ability of the model.

[0098] Adaptive learning rate can improve the robustness of intelligent agents in dynamic environments, enabling them to better cope with challenges brought by environmental changes.

[0099] Balancing short-term and long-term gains: The discount factor can balance the short-term and long-term gains of the intelligent agent in a dynamic environment, enabling it to pursue short-term goals while also considering long-term goals.

[0100] The discount factor can help intelligent agents better evaluate the long-term value of different decisions and thus make higher-quality decisions.

[0101] The discount factor can improve the adaptability of intelligent agents in dynamic environments, enabling them to better cope with the challenges brought by environmental changes.

[0102] In Example 3, the Q value function calculation formula is:

[0103] ;

[0104] in Indicates that the i-th intelligent agent is in the local state and neighbor status Next, select Action Q value;

[0105] is the learning rate, ranging from ; is the immediate reward, indicating that the intelligent agent i is performing an action Direct rewards received after is the discount factor, ranging from ; Indicates the next local state of the intelligent agent i and neighbor local states Select Action The maximum Q value of is the local state of the intelligent agent i at the next moment; is the local state of the neighbors of the i-th intelligent agent at the next moment.

[0106] In Example 4, step 5, the consistency protocol formula is:

[0107] ;

[0108] in Represents the local state of intelligent agent i; The local state of the neighboring agent j of agent i; represents the neighbor set of intelligent agent i, which is determined by the communication range and task requirements; is a normalization factor used to calculate the average of the local states of agent i and its neighbors.

[0109] In embodiment 5, the local state is position, velocity and sensor data.

[0110] In Example 6, the action selection formula in step 3 is:

[0111] ;

[0112] in represents the action selected by intelligent agent i; is the set of actions of intelligent agent i; Indicates that the intelligent agent i is in the local state and neighbor local states Select Action Q value.

[0113] In embodiment 7, the actions are moving direction, speed adjustment and task allocation.

[0114] Embodiment 8, a multi-intelligent agent collaborative interaction system, used to implement a multi-intelligent agent collaborative interaction method, the multi-intelligent agent collaborative interaction system includes multiple intelligent agents and a communication network module 2, the intelligent agent includes an intelligent motion body 11 and a collaborative interaction module 12, the collaborative interaction module 12 is installed on the intelligent motion body 11 and is communicatively connected to the main control chip of the intelligent motion body 11, the collaborative interaction module 12 includes an interactive wireless communication module 121, a computing module 122 and a sensor module 123, the interactive wireless communication module 121 is wirelessly connected to the interactive wireless communication module 121 of the adjacent intelligent agent based on the local network communication formed by the communication network module 2, the computing module 122 is communicatively connected to the interactive wireless communication module 121 and the sensor module 123, and the sensor module 123 is used to monitor the local state of the intelligent motion body 11.

[0115] In Example 9, the communication network module 2 forms a local network communication within the operating range of multiple intelligent sports bodies 11 based on ZigBee or Lora wireless communication technology, and the interactive wireless communication module 121 is wirelessly connected with the interactive wireless communication module 121 of the adjacent intelligent agent body based on ZigBee or Lora wireless communication.

[0116] In Example 10, the computing module 122 includes a cache 1221 and a processor 1222, the cache 1221 is communicatively connected to the interactive wireless communication module 121, the processor 1222 is communicatively connected to the cache 1221, and the sensor module 123 includes a camera 1231, a laser radar 1232 and an inertial measurement unit 1233, and the camera 1231, the laser radar 1232 and the inertial measurement unit 1233 are respectively communicatively connected to the cache 1221.

[0117] Intelligent agents 11 may include robots, drones, and the like. The collaborative interaction module enables real-time interaction between these agents, improving collaborative efficiency. The sensor module acquires environmental information surrounding these agents, providing a basis for their decision-making. This sensor module enhances the agents' perception capabilities, enabling them to better adapt to environmental changes.

[0118] ZigBee or Lora wireless communication technology has the characteristics of low power consumption and can extend the working time of the intelligent agent.

[0119] The following uses a specific embodiment to illustrate the implementation principle of a multi-intelligent agent collaborative interaction method and system of the present invention:

[0120] The system consists of multiple mobile robots whose goal is to collaboratively carry objects to designated locations in a dynamic environment.

[0121] Scenario description:

[0122] Mission objective: Multiple robots collaborate to carry a large object to the target location.

[0123] -Environmental characteristics: There are obstacles in the environment, and the positions of the obstacles may change dynamically.

[0124] Robot capabilities: Each robot is equipped with sensors (such as cameras, lidar), communication modules and computing units, and can perceive the local environment and communicate with other robots.

[0125] Proxy Definition

[0126] Define a collection of robots, each of which is an intelligent agent.

[0127] The local state of each robot includes: its own position, target position, and obstacle information detected by sensors.

[0128] The action space of each robot includes: moving direction (forward, backward, left, right) and moving speed (slow, medium, high).

[0129] Each robot communicates with surrounding robots through a wireless communication module.

[0130] The neighbor set is defined as other robots within the communication range of the robot.

[0131] Each robot periodically broadcasts its own status and receives the status of its neighbors.

[0132] Reward function design:

[0133] The instant reward design is as follows:

[0134] If the robot moves towards the goal position, the reward is +1.

[0135] If the robot is far away from the target position, the reward is -1.

[0136] If the robot collides with an obstacle or a neighbor robot, the reward is -10.

[0137] If the robot reaches the goal position, reward +100.

[0138] Specific implementation steps:

[0139] Initialization: Each robot initializes its own state and Q-value function.

[0140] Set the learning rate and discount factor.

[0141] Local state observation: Each robot obtains local state through sensors: its own position, target position, obstacle position information. It obtains neighbor state through communication module.

[0142] Distributed decision making:

[0143] Each robot selects an action based on its local state and neighbor states using a Q-value function:

[0144] - For example, a robot might select "Move forward (medium speed)" as an action.

[0145] Local communication and coordination: Each robot broadcasts its chosen action to its neighboring robots.

[0146] The neighbor robot adjusts its own Q value function based on the received action information:

[0147] For example, if the robot chooses "Move forward (medium speed)" and receives a reward of +1, its Q-value is updated.

[0148] Global consistency guarantee: Each robot updates its local state using a consensus protocol:

[0149] For example, the robots adjust their target positions based on their neighbors' position information to ensure that all robots work together to carry objects.

[0150] Each robot periodically updates its local state and neighbor states.

[0151] If a new obstacle appears in the environment, the robot adjusts the Q-value function and reselects the action.

[0152] Hardware design:

[0153] Communication module:-Use low-power wireless communication module (such as ZigBee or LoRa) to support multi-hop communication.

[0154] Each robot is equipped with a communication module, and the communication range is set to 10 meters.

[0155] Computing Module: Each robot is equipped with an embedded processor (such as an ARM Cortex-M4) that runs reinforcement learning algorithms and consensus protocols. The processor has 256KB of memory, supporting real-time decision-making.

[0156] Sensor module: Each robot is equipped with the following sensors:

[0157] LiDAR: used to detect obstacles; Camera: used to identify target location; Inertial Measurement Unit (IMU): used for positioning and navigation.

[0158] Power module: Each robot is equipped with a rechargeable lithium battery with a capacity of 5000mAh. It supports solar charging and has a battery life of up to 8 hours.

[0159] Task completion rate: In a dynamic environment, the system task completion rate reaches more than 90%.

[0160] Communication overhead: Local communication reduces communication overhead by 60%.

[0161] Dynamic environmental adaptability: The system can respond to environmental changes and replan the path within 5 seconds.

[0162] This example demonstrates a multi-robot collaborative handling system based on distributed reinforcement learning and local communication. Through Q-value updates, a consensus protocol, and an adaptive learning mechanism, the system effectively addresses communication bottlenecks, global consistency, and adaptability to dynamic environments. The hardware design ensures the system's real-time performance and robustness. This approach can be extended to other multi-agent collaborative tasks, such as intelligent transportation and drone formations.

[0163] The above are all preferred embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-agent collaborative interaction method, characterized by: The following steps are involved: Step 1: Define a set of intelligent agents and initialize the local state and action space of each intelligent agent; Step 2: Each intelligent agent observes its local state and obtains the local state information of its neighboring intelligent agents through local network communication. Step 3: Each intelligent agent uses a reinforcement learning algorithm to select an action based on its local state and the local states of its neighbors. Step 4: The intelligent agent broadcasts the selected action information to neighboring intelligent agents through local network communication. The neighboring intelligent agents adjust their own Q-value functions according to the received action information. Step 5: Introduce a consensus protocol to ensure that local decisions among intelligent agents tend to be consistent; Step 6: Each intelligent agent periodically updates its local state and neighboring local states, and adjusts the Q-value function according to environmental changes. In step 5, the consistency protocol formula is: ； in Represents the local state of intelligent agent i; The local state of the neighboring agent j of agent i; represents the neighbor set of intelligent agent i, which is determined by the communication range and task requirements; is a normalization factor used to calculate the average value of the local state of the intelligent agent i and its neighbors; In step 6, the Q-value function is adjusted to introduce adaptive learning rate and discount factor to improve the adaptability of multi-agent collaborative interaction in dynamic environments; The Q value function calculation formula is: ； in Indicates that the i-th intelligent agent is in the local state and neighbor status Next, select Action Q value; is the learning rate, ranging from ; is the immediate reward, indicating that the intelligent agent i is performing an action Direct rewards received after is the discount factor, ranging from ; Indicates the next local state of the intelligent agent i and neighbor local states Select Action The maximum Q value of is the local state of the intelligent agent i at the next moment; is the local state of the neighbors of the i-th intelligent agent at the next moment; The local state is position, velocity, and sensor data.

2. The multi-agent collaborative interaction method according to claim 1, characterized in that: The action selection formula in step 3 is: ； in represents the action selected by intelligent agent i; is the set of actions of intelligent agent i; Indicates that the intelligent agent i is in the local state and neighbor local states Select Action Q value.

3. The multi-agent collaborative interaction method according to claim 2, characterized in that: Actions are movement direction, speed adjustment, and task allocation.

4. A multi-intelligent agent collaborative interaction system, characterized by: A method for implementing a multi-intelligent agent collaborative interaction method as described in claim 3, wherein the multi-intelligent agent collaborative interaction system includes a plurality of intelligent agents and a communication network module (2), wherein the intelligent agents include an intelligent motion body (11) and a collaborative interaction module (12), wherein the collaborative interaction module (12) is installed on the intelligent motion body (11) and is communicatively connected to the main control chip of the intelligent motion body (11), wherein the collaborative interaction module (12) includes an interactive wireless communication module (121), a computing module (122) and a sensor module (123), wherein the interactive wireless communication module (121) is wirelessly connected to the interactive wireless communication module (121) of an adjacent intelligent agent based on the local network communication formed by the communication network module (2), wherein the computing module (122) is communicatively connected to the interactive wireless communication module (121) and the sensor module (123), and wherein the sensor module (123) is used to monitor the local state of the intelligent motion body (11).

5. The multi-agent collaborative interaction system according to claim 4, characterized in that: The communication network module (2) forms a local network communication within the operating range of the plurality of intelligent moving bodies (11) based on ZigBee or Lora wireless communication technology, and the interactive wireless communication module (121) is wirelessly connected to the interactive wireless communication module (121) of the adjacent intelligent agent body based on ZigBee or Lora wireless communication.

6. The multi-agent collaborative interaction system according to claim 5, characterized in that: The computing module (122) includes a buffer (1221) and a processor (1222), the buffer (1221) is communicatively connected to the interactive wireless communication module (121), the processor (1222) is communicatively connected to the buffer (1221), and the sensor module (123) includes a camera (1231), a laser radar (1232), and an inertial measurement unit (1233), and the camera (1231), the laser radar (1232), and the inertial measurement unit (1233) are respectively communicatively connected to the buffer (1221).

Images