Energy-saving multi-agent exploration system based on deep reinforcement learning

Abstract

The invention belongs to the field of artificial intelligence, multi-agent systems, deep reinforcement learning and autonomous exploration, and discloses an energy-saving multi-agent exploration system based on deep reinforcement learning, which comprises scene modeling, an efficiency-oriented observation system, an inter-agent strategy network, a sequence decision mechanism module and a waiting mechanism. According to the efficiency-oriented observation system provided by the invention, a dual structure of a connection diagram and an interaction diagram is constructed, and multi-dimensional cooperation-oriented features such as exploration values, distance features and complexity information are added to nodes, so that the system can accurately grasp the interaction relationship between environment features and agents, and the cooperation efficiency of a multi-agent system is remarkably improved. The sequence decision-making mechanism module designed by the invention combines a sequence decision-making mechanism and a waiting mechanism, and energy waste caused by blind cooperation is avoided by enabling the agent to refer to the decision of the preorder agent and select waiting at a proper time.

Description

Technical Field

[0001] The present invention belongs to the fields of artificial intelligence, multi-agent systems, deep reinforcement learning and autonomous exploration, and relates to an energy-saving multi-agent exploration system based on deep reinforcement learning, specifically an energy-saving multi-agent exploration system, which combines efficiency-oriented observation, inter-agent strategy network and waiting mechanism to improve the collaborative exploration efficiency of multi-agents in unknown environments. Background Art

[0002] With the development of artificial intelligence and robotics, multi-agent systems have shown great application potential in the exploration of unknown environments and have been widely used in target transportation, security monitoring, disaster detection, and environmental management. Multi-agent exploration tasks require multiple agents to collaboratively explore in partially observable unknown environments, build environmental maps, and complete designated tasks. This technology has important practical value in many scenarios such as smart cities, emergency rescue, and deep-sea exploration.

[0003] At present, multi-agent exploration methods are mainly divided into three categories: boundary-based methods, path planning-based methods, and deep reinforcement learning-based methods. Boundary-based methods guide the exploration direction by identifying the boundaries between known and unknown areas, and have the advantage of being simple to implement. Path planning-based methods (such as RRT, PRM, etc.) focus on achieving regional coverage while ensuring path feasibility. In recent years, with the breakthrough of deep learning technology, methods based on deep reinforcement learning (such as MAPPO, QMIX, etc.) have begun to emerge in multi-agent exploration tasks, achieving agent strategy optimization through end-to-end training.

[0004] Although deep reinforcement learning-based methods can use learned strategies to achieve intelligent exploration, existing methods are mostly limited to the perception of local information and the optimization of short-term goals. There are still deficiencies in the design of intelligent agent collaboration mechanisms, making it difficult to achieve global and efficient collaboration. In addition, in complex environments, due to the limitations of distributed decision-making, existing methods are prone to repeated exploration and redundant movement, further increasing the total path length and energy cost of exploration.

[0005] Therefore, how to improve exploration efficiency while reducing redundant exploration and energy consumption, and design a more efficient and robust agent collaboration mechanism, has become a core issue that needs to be solved in the field of multi-agent exploration. The solution to these challenges will directly affect the promotion and deployment of multi-agent exploration technology in practical applications. Summary of the invention

[0006] The present invention aims to provide a multi-agent energy-saving autonomous exploration system based on deep reinforcement learning, aiming to solve problems such as path redundancy caused by insufficient collaboration in existing exploration algorithms. By proposing efficiency-oriented observations to accurately model the environment and extract interaction information, the collaboration between agents is strengthened and redundant exploration is avoided. Combined with the inter-agent strategy network, the ability of the entire system to understand the environment is deepened. In addition, the novel waiting mechanism further avoids additional exploration behavior. The present invention has made good improvements in reducing exploration overlap and saving energy costs for multi-agent exploration problems.

[0007] The technical solution of the present invention:

[0008] An energy-saving multi-agent exploration system based on deep reinforcement learning, the energy-saving multi-agent exploration system includes scenario modeling, efficiency-oriented observation system, inter-agent strategy network, sequential decision mechanism module and waiting mechanism;

[0009] (1) Scenario Modeling

[0010] The scene to be explored is defined as Two-dimensional occupancy grid map of , which contains the explored area and unexplored areas ,satisfy ; The explored area is further divided into free areas and occupied area ,satisfy ; Each agent is equipped with a detection range of Laser scanner to update explored areas ; At the beginning of the exploration, ; The optimization goal of the energy-saving multi-agent exploration system is to find the shortest agent trajectory To complete the exploration process of the entire area while ensuring energy efficiency and collaborative performance during the exploration process;

[0011] (2) Building an efficiency-oriented observation system

[0012] Enhance overall situational awareness and emphasize the interaction between agents by first building a connection graph , used to characterize the connectivity of the environment; in the explored area Uniform sampling in the middle, establish a node set , and based on the collision-free path for each node and its The nearest neighbor nodes establish an edge set ; Connection diagram Over time Dynamic update, the agent is updated according to the connection graph The features of neighbor nodes in the network select the next action; the main function of the connection graph is to build the topological structure of the environment to ensure that the agent can efficiently cover the unknown area during the exploration process; in order to strengthen the collaborative relationship between agents, the interaction graph is further constructed. , its node set and connection graph Consistent, edge set A logical undirected graph is formed to dynamically connect the current position node of the agent and the target node with a non-zero exploration value (the number of observable exploration boundaries at this node); an interaction graph Emphasizes collaboration between agents and promotes the effectiveness of collaborative exploration by dynamically connecting high-utility nodes and other agent positions; edge collection The construction in time Dynamically capture the collaboration needs between agents;

[0013] Among the features that characterize the exploration value of each node, complexity information It is a key feature used to quantify the exploration difficulty and potential collaboration requirements of the region; complexity information The density clustering algorithm (DBSCAN) is used to cluster the boundary points corresponding to the nodes, which reflects the bifurcation structure and exploration complexity in the region. Complexity information is defined by these characteristics. A higher complexity value indicates that the region may contain more branches, which is more difficult to explore and requires stronger collaboration of the agents to complete the task. For example, in a single path region, the complexity information The value of is usually low, while in the region of bifurcations or crossing paths, the complexity information The value of will increase significantly, suggesting that the agents need to collaborate to allocate tasks. Complexity information The calculation method of is as follows: first, the boundary points are extracted from the perception range of each node. The boundary points are the boundaries between the explored area and the unknown area. Through the density clustering algorithm, the distribution of the boundary points is divided into different clusters to reflect the geometric characteristics of the area where the node is located, including the number of bifurcations, the spacing between bifurcations, and the angle of bifurcations. Finally, the number of clusters output by the density clustering algorithm is directly used as the complexity information. The value of

[0014] The introduction of complexity information makes up for the limitations of relying solely on exploration value. Exploration value mainly characterizes the number of unknown areas in the region, but cannot effectively identify the potential collaboration needs of complex bifurcation scenarios. Through complexity information, the energy-saving multi-agent exploration system can identify key areas with high geometric complexity and reasonably allocate agent resources, thereby avoiding redundant paths and resource waste caused by simply pursuing high exploration values. Complexity information can also dynamically adjust the agent's behavior strategy, enabling it to complete exploration tasks more accurately in complex scenarios, significantly improving the overall exploration efficiency.

[0015] (3) Construct an inter-agent policy network to improve the cognitive and decision-making capabilities of the agent through the attention mechanism, including two encoders and a decoder ;

[0016] Two encoders In parallel from the connection graph and interaction diagrams Extract node information from Each layer consists of 6 layers of multi-head self-attention layers, where the output of the upper multi-head self-attention layer becomes the input of the lower multi-head self-attention layer; the input of each multi-head self-attention layer includes the query vector , key vector Sum value vector , by learning the matrix , , generate, , , Iterable parameters for random initialization; weight By query vector With key vector The dot product is calculated and scaled by After normalization, it is processed by the softmax function; to ensure that there is no information transmission between nodes without edge connections, a mask matrix is ​​introduced , when the node and nodes When there is an edge ,otherwise ; Connection diagram and interaction diagrams After passing through two encoders in parallel, the output and are stitched and projected onto dimensional feature space to generate enhanced node features ;

[0017] Decoder It consists of an attention layer and a pointer layer, and generates action probability distribution through multi-stage processing; first, it enhances node features Extract the current node features and neighbor characteristics , the previous node feature As a query, enhance node features Input the attention layer as key and value; the output generated by the attention layer is the same as the previous node feature After stitching, the projection is the current enhanced feature ; Finally, the action probability distribution is generated through the pointer layer, where the current enhanced feature As a query, As keys and values; the pointer layer dynamically adjusts the action space and outputs the strategy distribution , indicating the time The probability of selecting the next node is Indicates time The corresponding environmental observations, Respectively indicate time Decision points and moments Node and time All neighboring nodes of the node;

[0018] (4) Sequential decision mechanism module, which decomposes the actions of multiple agents into a series of orderly decision-making processes;

[0019] In order to solve the problems of unstable behavior and inefficient collaboration in multi-agent exploration, the sequential decision mechanism module proposes a sequential decision mechanism, which innovatively decomposes the actions of multi-agents into a series of orderly decision-making processes, thereby significantly improving the collaborative efficiency. Although the decisions of the energy-saving multi-agent exploration system are executed under a synchronous framework, by introducing a sequential decision-making method, each agent can fully refer to the decision results of the previous agent when formulating its own actions, thereby forming a more collaborative and holistic exploration behavior on a global scale.

[0020] In the decoder In the input of the pointer layer, the agent Current Enhancements is further enhanced by the preceding action, where For the intelligent agent , the energy-saving multi-agent exploration system At the moment Selected enhanced target node feature sequence With decoder Output current enhancement node features To splice, here, For the Intelligent , ; The concatenated features are passed through a dimension of The 2D convolutional layer reduces it back to a uniform dimension, where represents the total number of agents in the energy-saving multi-agent exploration system, represents the feature dimension; finally, the output of the convolutional layer is used as the new The input pointer layer is used to integrate the previous action information. The design of the convolutional layer can efficiently integrate the action sequence characteristics between agents while retaining the independence and collaboration between individuals, thereby improving the integrity and robustness of the decision.

[0021] (5) Waiting mechanism, which dynamically adjusts the behavior state of the agent to achieve reasonable allocation of exploration resources and efficient collaboration among multiple agents;

[0022] In multi-agent exploration tasks in complex scenarios, due to the highly uneven distribution of environments and tasks, the exploration progress of different agents often varies significantly. Some agents may complete the exploration tasks of their responsible areas earlier than other agents. If these agents blindly continue to collaborate or try to explore other areas after completing their tasks, it usually leads to unnecessary movement and energy waste. In order to solve this problem, the present invention proposes a waiting mechanism, which dynamically adjusts the behavior state of the agents to achieve reasonable allocation of exploration resources and efficient collaboration of multiple agents.

[0023] When an agent completes the exploration task of the area it is responsible for, the energy-saving multi-agent exploration system conducts real-time evaluation of its current state and the exploration task status of other agents; by adding a "wait" action in the action space of the agent, it allows it to judge whether it needs to continue the exploration behavior based on the current observation; the energy-saving multi-agent exploration system dynamically judges whether the agent needs to continue to participate in the collaboration or remain stationary by sensing the task progress, regional complexity and target node distribution of other agents; if the exploration tasks of other agents in their respective areas have not yet reached the level that requires external collaboration, the energy-saving multi-agent exploration system will keep the agent that has completed the task stationary to avoid meaningless node selection or area movement; on the contrary, when other agents encounter exploration bottlenecks due to task difficulty or regional complexity, the agent that has completed the task is dynamically activated to continue to participate in the collaboration.

[0024] The implementation of the waiting mechanism relies on the real-time perception and dynamic evaluation of the multi-agent state by the energy-saving multi-agent exploration system. The agents that have completed the task will not directly enter a completely static state, but will be in a standby mode that can be activated at any time. When the system perceives new task requirements or other agents encounter exploration difficulties, these standby agents can quickly resume their active state and re-participate in the exploration task. At the same time, by continuously monitoring the task status, the system can dynamically adjust the behavior of the agent to always match the current task requirements.

[0025] Beneficial effects of the present invention:

[0026] (1) The efficiency-oriented observation system proposed in the present invention constructs a dual structure of a connection graph and an interaction graph, and adds multi-dimensional collaboration-oriented features such as exploration value, distance feature, and complexity information to nodes, so that the system can accurately grasp the environmental characteristics and the interaction relationship between intelligent agents, and significantly improve the collaborative efficiency of the multi-agent system.

[0027] (2) The sequence decision mechanism module designed by the present invention innovatively combines the sequence decision mechanism and the waiting mechanism. By allowing the agent to refer to the decision of the previous agent and choose to wait at the appropriate time, it avoids the energy waste caused by blind cooperation. Tests show that while maintaining the same exploration coverage, the system significantly improves the exploration efficiency of the system in large-scale complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Flowchart of the pipeline for the multi-agent exploration method.

[0029] Figure 2 is a node-connected graph.

[0030] Figure 3 It is the interaction diagram between nodes.

[0031] Figure 4 This is an exploration case without the introduction of interaction diagrams.

[0032] Figure 5 This is an exploration case after the introduction of the interaction diagram. DETAILED DESCRIPTION

[0033] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0034] Example 1

[0035] A multi-agent energy-saving autonomous exploration system based on deep reinforcement learning (the overall process is as follows Figure 1 As shown), the steps are as follows:

[0036] (1) First, an efficiency-oriented observation system is used to accurately represent the observation characteristics of the environment.

[0037] For a specific complex real scene, the efficiency-oriented observation system uniformly samples 1024 points in the explored free area as the vertex set, and establishes collision-free connections between each vertex and its nearest 20 neighbors to construct a connection graph, such as Figure 2 To enhance the interactive perception between intelligent agents, the present invention adds exploration value, distance feature, complexity information and access feature to each node.

[0038] The exploration value and distance features are obtained by calculating the number of observable boundaries and the Euclidean distance at the node, respectively. For the calculation of complexity information, the observable boundary point set of the current node is first detected, and the nodes with observable boundary points greater than zero are defined as points with exploration potential. Subsequently, the density-based clustering algorithm DBSCAN is used to perform cluster analysis on the boundary points. Specifically, for each boundary point, the number of points in its neighborhood is calculated with ε as the radius, and when the number of points exceeds the threshold, it is marked as a core point. By iteratively classifying all density-reachable boundary points into one category, k clusters are finally obtained. The complexity information is determined by the number of clusters k. The access feature is binary marked according to whether the node has been visited.

[0039] When constructing the interaction graph, the system maintains the same node set and node features as the connection graph, but uses a manually defined edge set to highlight the collaborative relationship between agents. This edge set connects all the nodes where the agents are located with points with exploration potential to ensure the smooth flow of exploration information. The schematic diagram is shown below: Figure 3 As shown in Figure 2, the efficiency-oriented observation module integrates the features of the connection graph and the interaction graph and outputs them, providing a structured environmental representation for the subsequent policy network. This multi-level feature extraction mechanism based on the graph provides the necessary environmental perception information for the collaborative decision-making of the intelligent agent. Figure 4 and Figure 5 The effect of adding an interactive graph on exploration performance is shown in Figure 1. From left to right, the exploration cases without and with interactive graphs are shown. In the figure on the left, the exploration routes of multiple agents overlap, resulting in a large amount of redundant exploration, while in the figure on the right, the exploration routes of each agent complete the exploration of the entire map with a low overlap rate.

[0040] (2) The inter-agent policy network enhances the cognitive and decision-making capabilities of intelligent agents through the inter-agent attention mechanism.

[0041] The inter-agent policy network enhances the cognitive and decision-making capabilities of the agent through the inter-agent attention mechanism. In the feature encoding stage, the system first starts two encoders in parallel to process the connection graph and the interaction graph respectively. Each encoder consists of 6 multi-head self-attention layers stacked in sequence. The attention layer adaptively extracts important structural information in the graph by calculating the similarity between the query vector and the key vector and applying the obtained attention weight to the value vector. In the feature fusion stage, the outputs of the two encoders are integrated to form enhanced node features. The system extracts node features representing the current position information of the agent and adjacent node features describing the surrounding environment from the enhanced node features, which are then input into the decoder for processing. The decoder also uses the attention mechanism to generate a contextual representation that integrates local and global information by modeling the correlation between the current position feature and the adjacent features. In the action generation stage, the output of the decoder is processed by the pointer layer, which first calculates the correlation score between the contextual representation and the candidate action, and then converts these scores into probability distributions through the softmax function. The entire processing flow is trained end-to-end through backpropagation, allowing the system to learn efficient feature extraction and decision-making strategies. Among them, the features are encoded into 128 dimensions, and the multi-head attention layer is set to 8 heads for learning.

[0042] (3) Avoid unstable actions and redundant exploration by serializing decision-making mechanisms and waiting mechanisms.

[0043] The sequential decision mechanism module avoids unstable actions and redundant exploration through the sequential decision mechanism and waiting mechanism. In the sequential decision processing stage, the system processes the decision of each agent in turn according to the preset order (1 to N). , first collect the previous agent 1 to The selected enhanced target node feature set is concatenated with the enhanced node features of the current agent to form an extended feature vector. The extended feature vector is then processed by a 2D convolutional layer. The output of the convolutional layer is processed by a nonlinear activation function to generate fused features that reflect the collaborative relationship between agents.

[0044] In the action space construction phase, the system constructs a hybrid action space for each agent that contains two types of actions: movement and waiting. The movement action set contains the neighboring nodes of the agent's current position, indicating the possible moving target position of the agent. At the same time, the system considers the waiting action candidate as an action vector, so that the agent can choose to remain still when appropriate. The design of this hybrid action space enables the system to flexibly respond to different exploration scenarios.

[0045] In the policy network prediction phase, the fused features are input into the policy network consisting of a multi-layer neural network. The policy network first transforms the features through a fully connected layer, and then uses an attention mechanism to enhance the extraction of important information. Finally, the probability distribution in the mixed action space is output through the softmax layer, and sampling is used to decide the execution action. This end-to-end training method enables the system to adaptively learn when to move and when to wait, thereby effectively avoiding redundant exploration and energy waste. The entire decision-making process is optimized through experience replay and SAC methods to maximize long-term exploration efficiency.

[0046] (4) Use collaborative reward mechanisms to encourage exploratory behavior

[0047] In order to further improve the exploration efficiency of the agent and encourage collaborative behavior, this paper uses an exploration incentive method based on a collaborative reward mechanism. This mechanism guides the agent to adopt a more globally valuable exploration strategy through a carefully designed reward function, thereby improving the exploration efficiency and optimizing resource utilization. The reward function consists of three parts: exploration reward, distance penalty, and completion reward.

[0048] During the agent's exploration process, the exploration reward mainly measures the amount of newly discovered environmental information, that is, the unknown area newly explored by the agent in each step of action. This reward encourages the agent to move towards unvisited areas to maximize the overall environmental coverage. At the same time, considering resource utilization, the system introduces a distance penalty to constrain the cost of the agent's movement from the current position to the next position. This penalty term ensures that the agent does not perform too long or unnecessary movements during the exploration process, thereby saving energy and improving the economy of action. In addition, in order to further motivate the agent to complete the entire exploration task, the system introduces a completion reward. When all areas to be explored are completely covered, the agent will receive a fixed reward. This reward is only given when the task is completed to ensure the final achievement of the exploration goal.

[0049] Ultimately, the entire reward function, through a weighted combination of exploration rewards, distance penalties, and completion rewards, enables the agent to reduce redundant actions and optimize resource consumption while ensuring efficient exploration, thereby improving the overall task completion efficiency.

Claims

1. An energy-saving multi-agent exploration system based on deep reinforcement learning, characterized in that: The energy-saving multi-agent exploration system includes scenario modeling, efficiency-oriented observation system, inter-agent strategy network, sequential decision mechanism module and waiting mechanism; (1) Scenario Modeling The scene to be explored is defined as Two-dimensional occupancy grid map of , which contains the explored area and unexplored areas ,satisfy ; The explored area is further divided into free areas and occupied area ,satisfy ; Each agent is equipped with a detection range of Laser scanner to update explored areas ; At the beginning of the exploration, ; The optimization goal of the energy-saving multi-agent exploration system is to find the shortest agent trajectory To complete the exploration process of the entire area while ensuring energy efficiency and collaborative performance during the exploration process; (2) Building an efficiency-oriented observation system First build the connection graph , used to characterize the connectivity of the environment; in the explored area Uniform sampling in the middle, establish a node set , and based on the collision-free path for each node and its The nearest neighbor nodes create an edge set ; Connection diagram Over time Dynamic update, the agent is updated according to the connection graph The features of neighbor nodes in the middle select the next action; to strengthen the collaborative relationship between agents, further build an interaction graph , its node set and connection graph Consistent, edge set To dynamically connect the agent's current position node and the target node with a non-zero exploration value to form a logical undirected graph; Among the features that characterize the exploration value of each node, complexity information It is a key feature used to quantify the exploration difficulty and potential collaboration requirements of the region; complexity information The density clustering algorithm is used to cluster the boundary points corresponding to the nodes, which reflects the bifurcation structure and exploration complexity in the region; (3) Construct an inter-agent policy network to improve the cognitive and decision-making capabilities of the agent through the attention mechanism, including two encoders and a decoder ; Two encoders In parallel from the connection graph and interaction diagrams Extract node information from Each layer consists of 6 layers of multi-head self-attention layers, where the output of the upper multi-head self-attention layer becomes the input of the lower multi-head self-attention layer; the input of each multi-head self-attention layer includes the query vector , key vector Sum value vector , by learning the matrix , , generate, , , Iterable parameters for random initialization; weight By query vector With key vector The dot product is calculated and scaled by After normalization, it is processed by the softmax function; to ensure that there is no information transmission between nodes without edge connections, a mask matrix is ​​introduced , when the node and nodes When there is an edge ,otherwise ; Connection diagram and interaction diagrams After passing through two encoders in parallel, the output and are stitched and projected onto dimensional feature space to generate enhanced node features ; Decoder It consists of an attention layer and a pointer layer, and generates action probability distribution through multi-stage processing; first, it enhances node features Extract the current node features and neighbor characteristics , the previous node feature As a query, enhance node features Input to the attention layer as key and value; Output generated by the attention layer and features of the previous node After stitching, the projection is the current enhanced feature ; Finally, the action probability distribution is generated through the pointer layer, where the current enhanced feature As a query, As keys and values; the pointer layer dynamically adjusts the action space and outputs the strategy distribution , indicating the time The probability of selecting the next node is Indicates time The corresponding environmental observations, Respectively indicate time Decision points and moments Node and time All neighboring nodes of the node; (4) Sequential decision mechanism module, which decomposes the actions of multiple agents into a series of orderly decision-making processes; In the decoder In the input of the pointer layer, the agent Current Enhancements is further enhanced by the preceding action, where For the intelligent agent , the energy-saving multi-agent exploration system At the moment Selected enhanced target node feature sequence With decoder Output current enhancement node features To splice, here, For the Intelligent , ; The concatenated features are passed through a dimension of The 2D convolutional layer reduces it back to a uniform dimension, where represents the total number of agents in the energy-saving multi-agent exploration system, represents the feature dimension; finally, the output of the convolutional layer is used as the new Input pointer layer to integrate the previous action information; (5) Waiting mechanism: By dynamically adjusting the behavior state of the intelligent agent, it can achieve the rational allocation of exploration resources and the efficient collaboration of multiple intelligent agents.

2. The energy-saving multi-agent exploration system based on deep reinforcement learning according to claim 1 is characterized in that: In step (2), the complexity information The calculation method of is as follows: first, the boundary points are extracted from the perception range of each node. The boundary points are the boundaries between the explored area and the unknown area. Through the density clustering algorithm, the distribution of the boundary points is divided into different clusters to reflect the geometric characteristics of the area where the node is located, including the number of bifurcations, the spacing between bifurcations, and the angle of bifurcations. Finally, the number of clusters output by the density clustering algorithm is directly used as the complexity information. The value of .

3. The energy-saving multi-agent exploration system based on deep reinforcement learning according to claim 1 is characterized in that: The specific implementation process of step (5) is as follows: When an agent completes the exploration task of the area it is responsible for, the energy-saving multi-agent exploration system evaluates its current state and the exploration task status of other agents in real time; by adding a "wait" action in the action space of the agent, it allows it to judge whether it needs to continue the exploration behavior based on the current observation; the energy-saving multi-agent exploration system dynamically judges whether the agent needs to continue to participate in the collaboration or remain stationary by sensing the task progress, regional complexity and target node distribution of other agents; if the exploration tasks of other agents in their respective areas have not yet reached the level that requires external collaboration, the energy-saving multi-agent exploration system will keep the agent that has completed the task stationary to avoid meaningless node selection or regional movement; on the contrary, when other agents encounter exploration bottlenecks due to task difficulty or regional complexity, the agent that has completed the task is dynamically activated to continue to participate in the collaboration.

Images