Multi-agent navigation method and electronic device

Abstract

The application provides a multi-agent navigation method and an electronic device, and relates to the technical field of multi-agent control. The method comprises the following steps: acquiring state information and environment information of each agent; generating a global reference path conforming to the motion constraint of each agent based on the state information, the environment information and the physical parameters of each agent; generating a multi-dimensional feature vector according to the environment information and the global reference path; inputting the multi-dimensional feature vector into a decision network, and performing attention operations and weighted fusion on the features in the multi-dimensional feature vector based on a multi-head attention mechanism to obtain a local motion instruction; and controlling the motion of the agent according to the local motion instruction. The technical scheme of the application provides long-term navigation guidance through the global reference path, and makes local real-time decisions based on the decision network based on the multi-head attention mechanism, so as to balance the overall efficiency and dynamic obstacle avoidance demand, thereby realizing efficient, safe and self-adaptive multi-agent collaborative navigation in a complex dynamic environment.

Description

Technical Field

[0001] This application relates to the field of multi-agent control technology, and in particular to a multi-agent navigation method and electronic device. Background Technology

[0002] In the field of multi-agent control technology, traditional search algorithms and deep reinforcement learning methods have been widely adopted to ensure the effectiveness and real-time performance of path planning. In existing solutions, traditional search methods and deep reinforcement learning methods are often applied independently, with differentiated coordination among heterogeneous agents relying on manually set rules or simplified isomorphic assumptions. However, this separation or simplification approach cannot meet the high real-time and high safety requirements of heterogeneous multi-agent cooperative navigation in complex dynamic environments. Furthermore, fixed rules or isomorphic assumptions cannot adapt to the differences between agents with varying motion capabilities and task constraints, potentially leading to low planning efficiency, frequent cooperative conflicts, or difficulties in training convergence.

[0003] Therefore, there is an urgent need for a multi-agent path planning and navigation method that can balance global optimization and local real-time response, and adapt to the differences between heterogeneous agents. Summary of the Invention

[0004] This application provides a multi-agent navigation method and electronic device to solve the technical problems of existing technologies when performing multi-agent navigation in complex dynamic environments. These problems include difficulty in balancing global planning and local collaborative decision-making, inability to effectively handle the motion constraints and task differences of heterogeneous agents, resulting in low collaborative efficiency and susceptibility to deadlock or collision.

[0005] In a first aspect, embodiments of this application provide a multi-agent navigation method, including: Acquire the state and environmental information of each agent; Based on state information, environmental information, and the physical parameters of each agent, a global reference path that conforms to the motion constraints of each agent is generated. Generate multi-dimensional feature vectors based on environmental information and global reference paths; Multidimensional feature vectors are input into a decision network. The decision network uses a multi-head attention mechanism to perform attention operations on path guidance features, spatial obstacle features, temporal dynamic features, and resource competition features in the multidimensional feature vectors, and then weights and fuses the results to obtain the local motion instructions of each agent. The agent is controlled to move according to local motion commands.

[0006] Based on this scheme, a global reference path that conforms to the motion constraints of each agent is generated, and environmental information is fused with the global reference path to generate a multi-dimensional feature vector. Then, a multi-head attention mechanism is used to perform attention operations on four types of features: path guidance, spatial obstacles, temporal dynamics, and resource competition, and then weighted and fused. This achieves the combination of global planning and local decision-making, enabling heterogeneous multi-agent systems to balance the overall efficiency of path planning and the real-time obstacle avoidance requirements in dynamic environments, thereby improving the system's adaptability and robustness.

[0007] In some embodiments, generating a global reference path for each agent that conforms to its motion constraints includes: The obstacles in the environmental information are expanded based on the physical parameters of each intelligent agent; Based on the jump point search algorithm, a global reference path that conforms to the motion constraints of each agent is generated after the dilation process.

[0008] Based on this scheme, by expanding obstacles according to the physical parameters of each agent and generating a global reference path that conforms to motion constraints based on the jump point search algorithm, it is possible to plan collision-free and executable paths for agents of different sizes and motion constraints, effectively solving the problems of size adaptation and motion constraints in path planning for heterogeneous agents.

[0009] In some embodiments, generating a multidimensional feature vector includes: Based on waypoints located ahead of each agent's current position in the global reference path, calculate and normalize the relative coordinates and orientation angles of each waypoint relative to the agent to generate path guidance features; The space around each agent is divided into multiple sectors. Within each sector, the type of the nearest obstacle, as well as its distance and relative velocity with respect to the agent, are detected to generate spatial obstacle features. The historical state sequences of each agent are encoded using a recurrent neural network to extract motion trend information and generate temporal dynamic features. Identify key resource points in the map, encode the distance from each agent to the resource point, the resource point's occupancy status, and waiting queue information, and generate resource competition features.

[0010] Based on this scheme, by extracting waypoints from the global reference path to generate path guidance features, dividing the surrounding space into sectors to generate spatial obstacle features, using recurrent neural networks to extract temporal dynamic features, and encoding key resource points to generate resource competition features, the structured encoding of multi-dimensional information required for heterogeneous multi-agent navigation is realized. This provides rich and accurate environmental representations for the decision network, which is beneficial for subsequent collaborative decision-making.

[0011] In some embodiments, inputting multidimensional feature vectors into a decision network includes: Extract path guidance features, spatial obstacle features, temporal dynamic features, and resource competition features from multidimensional feature vectors; The path guidance features are input into the global guidance head in the decision network to evaluate the path intent conflicts between agents and generate the first evaluation result. The spatial obstacle features are input into the spatial relationship head in the decision network to assess the instantaneous spatial collision risk between the current agent and other agents and obstacles in the surrounding area, and generate a second assessment result. The temporal dynamic features are input into the temporal dynamic head of the decision network to predict the short-term interaction situation between the agents and generate the third evaluation result. The resource competition characteristics are input into the resource competition head in the decision network to schedule the order of spatial resource occupation by each agent and generate the fourth evaluation result. The first, second, third, and fourth evaluation results are weighted and fused to generate local motion commands.

[0012] Based on this scheme, by inputting four types of features into the global guidance head, spatial relationship head, temporal dynamic head and resource competition head respectively, corresponding evaluation results are generated and weighted and fused, realizing parallel processing and collaborative decision-making of path intent conflict, immediate collision risk, short-term interaction status and resource occupation order, thereby improving the coordination ability and decision-making efficiency of multi-agents in complex scenarios.

[0013] In some embodiments, during the training of the decision network, the agent's behavior is evaluated based on a hierarchical reward function; the hierarchical reward function includes: Path guidance reward is calculated based on the degree of deviation between the agent's current position and the global reference path, in order to guide the agent to follow the global reference path; The safety obstacle avoidance reward is calculated based on the distance between the agent and surrounding obstacles, and a risk penalty coefficient is set according to the physical parameters of each agent to guide the agent to maintain a safe distance. The collaborative interaction reward is calculated based on the time taken by multiple agents to pass through key resource points and the weight of the task value, in order to guide multiple agents to collaborate through key resource points. Task progress rewards are calculated using a normalized ratio of the distance the agent travels to the target point to its maximum speed, in order to guide the agent toward the target point.

[0014] Based on this scheme, the hierarchical reward function achieves adaptive guidance for heterogeneous intelligent agents by setting risk penalty coefficients differently and normalizing task progress rewards. This effectively overcomes the shortcomings of sparse rewards and slow training convergence in traditional reinforcement learning, and significantly improves the learning efficiency and system stability of collaborative strategies.

[0015] In some embodiments, the method further includes dynamic replanning of the global reference path, specifically including: During the agent's movement, the first evaluation result output by the global guidance head is acquired in real time, and changes in environmental information are monitored. When a change in environmental information is detected, and / or when a path intent conflict is predicted between agents based on the first evaluation result and the risk exceeds a preset threshold, the affected agents whose paths are impassable or whose path intent conflicts exist due to environmental changes are identified, and a global reference path that conforms to their motion constraints is regenerated for the affected agents. The path-guided features in the multidimensional feature vector are updated based on the regenerated global reference path.

[0016] Based on this scheme, by monitoring environmental changes and path intent conflicts in real time, and identifying the affected agents, regenerating the global reference path and updating the path guidance features when the triggering conditions are met, adaptive replanning in dynamic environments is achieved. The replanning results are then fed back to the decision network in a closed loop, enhancing the system's safety and long-term navigation stability under emergencies.

[0017] In some embodiments, the resource contention head schedules the order in which agents occupy spatial resources, including: Use the resource competition characteristics of the current agent as the query, and the resource competition characteristics of other agents as the key and value; Based on the distance of each agent to the key resource point, the task priority, and the current occupancy status of the resource point, the attention weight is calculated. The agents are scheduled to form an orderly passage order based on attention weights.

[0018] Based on this scheme, by using the resource competition characteristics of the current intelligent agent as the query and the resource competition characteristics of other intelligent agents as the key and value, attention weights are calculated by comprehensively considering distance, task priority, and occupancy status, and the order of passage is scheduled accordingly. This achieves efficient collaborative occupation of scarce spatial resources such as intersections and narrow passages, effectively avoiding deadlock and congestion.

[0019] In some embodiments, a global reference path conforming to the motion constraints of each agent after dilation is generated based on a jump point search algorithm, including: In the path search process of the jump point search algorithm, a weighted heuristic function is used to evaluate the search nodes; the weighted heuristic function includes the actual path cost term, the distance heuristic term, and the task priority heuristic term. The value of the task priority heuristic is set according to the task priority of each agent, so that agents with high task priority will obtain lower heuristic values ​​during the search, thereby guiding the jump point search algorithm to prioritize the search towards the target point of agents with high task priority.

[0020] Based on this scheme, by adopting a weighted heuristic function that includes actual path cost, distance heuristic, and task priority heuristic in the jump point search algorithm, and setting the heuristic value according to the task priority, high-priority agents can obtain better paths, realizing global path planning with heterogeneous task priority awareness, and improving the navigation efficiency of task-critical agents.

[0021] In some embodiments, the decision network includes an execution network and an evaluation network, and the training process of the decision network includes: A centralized training mechanism is used to train the decision network, and the evaluation network obtains multi-dimensional feature vectors and local motion instructions from all agents. The evaluation network uses a multi-head attention mechanism to process and weight the multi-dimensional feature vectors of all agents, evaluates the fused features with the local motion instructions of all agents, and outputs an evaluation value to update the parameters of the execution network.

[0022] Based on this scheme, by adopting a centralized training mechanism during the training phase, the evaluation network uses a multi-head attention mechanism to weight and fuse the multi-dimensional feature vectors of all agents and jointly evaluate and output evaluation values ​​with action instructions. This achieves accurate evaluation of joint actions of multiple agents, effectively guides the parameter updates of the execution network, and improves the global optimality of the cooperative strategy.

[0023] Secondly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the aforementioned multi-agent navigation methods.

[0024] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0025] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0026] Figure 1 This illustration schematically shows an overall architecture diagram of a multi-agent navigation method according to an embodiment of this application; Figure 2 This illustration schematically shows a decision network model architecture of a multi-agent navigation method according to an embodiment of this application; Figure 3 This illustration schematically shows an overall flowchart of a multi-agent navigation method according to an embodiment of this application; Figure 4 This illustration schematically shows a global reference path generation process of a multi-agent navigation method according to an embodiment of this application; Figure 5 This illustration schematically shows a path generation flowchart of a jump point search algorithm for a multi-agent navigation method according to an embodiment of this application; Figure 6 This illustration schematically shows a multi-dimensional feature vector encoding process of a multi-agent navigation method according to an embodiment of this application; Figure 7 This illustration schematically shows a local motion command generation process of a multi-agent navigation method according to an embodiment of this application; Figure 8 This illustration schematically shows a hierarchical reward function evaluation process for a multi-agent navigation method according to an embodiment of this application; Figure 9 This illustration schematically shows a global reference path replanning process of a multi-agent navigation method according to an embodiment of this application; Figure 10 The illustration shows a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0028] The multi-agent navigation method provided in this application can be applied to various scenarios involving multi-agent collaborative operations. In these application scenarios, the devices involved typically include multiple agents with heterogeneous physical parameters and motion constraints, such as different types of automated guided vehicles, autonomous vehicles, or service robots, as well as a computing device for executing the navigation method of this application. The computing device can be an onboard computing unit deployed locally on the agent, or a central server in the cloud or at the edge. The agent can use its onboard sensors, such as LiDAR, cameras, inertial measurement units, and encoders, to collect its own state information and surrounding environment information in real time, and transmit this information to the computing device. After obtaining this information, the computing device can run the multi-agent navigation method provided in this application, calculate the local motion commands of each agent, and send the commands to the agent's motion control system, thereby controlling the agents to complete collaborative navigation tasks in a dynamically changing environment.

[0029] The following combination Figure 1-10 This application provides a detailed description of a multi-agent navigation method based on embodiments.

[0030] Figure 1 The diagram illustrates the overall architecture of a multi-agent navigation method according to an embodiment of this application.

[0031] like Figure 1 As shown in the overall architecture of this application embodiment, the input data 101 includes environmental information and the starting point and target point of each intelligent agent. The environmental information may include static map data, dynamic obstacle distribution, and the real-time positions of other intelligent agents, while the starting point and target point define the navigation task of each intelligent agent. This input data undergoes the following processing steps in sequence.

[0032] Global path planning (102) is performed, generating a global reference path for each agent based on a jump-point search algorithm, which conforms to its motion constraints. Specifically, this step expands the obstacles according to the physical parameters of each agent, such as vehicle size and minimum turning radius, thereby simplifying agents of different sizes into point masses and ensuring that the planned path is geometrically collision-free. Furthermore, a weighted heuristic function is used during the search process, which includes heuristics related to task priority, allowing higher-priority agents to obtain better paths.

[0033] Multidimensional feature vector encoding (103) is performed, generating a multidimensional feature vector based on environmental information and the global reference path generated in the previous step. This vector integrates features from multiple dimensions: path guidance features extracted from the global reference path, spatial obstacle features extracted from surrounding obstacle information, temporal dynamic features obtained by encoding historical state sequences through a recurrent neural network, and resource competition features extracted from key map resource points such as intersections and narrow passages. These features collectively provide a rich environmental representation for subsequent decision-making.

[0034] The decision network 104 utilizes a multi-head attention mechanism to process and weight-fuse multi-dimensional feature vectors, outputting local motion commands for each agent. Internally, the network contains four parallel attention heads, each specializing in path guidance features, spatial obstacle features, temporal dynamic features, and resource competition features. The evaluation results from each head are then weighted and fused to obtain control variables such as the linear velocity and angular velocity of the current agent. During the training phase, the network employs centralized training and a hierarchical reward function; during the execution phase, it uses only the execution network for distributed inference, thus achieving efficient and safe real-time navigation in dynamic environments.

[0035] Figure 2The diagram illustrates a decision network model architecture of a multi-agent navigation method according to an embodiment of this application.

[0036] like Figure 2 As shown, the decision network model architecture of this application embodiment includes: an agent information acquisition layer 201, a multi-head attention mechanism processing layer 202, a feature fusion layer 203, and a decision network layer 204.

[0037] In the agent information acquisition layer 201, the system acquires the state information and environmental information of each agent as the initial input to the decision network.

[0038] In the multi-head attention mechanism processing layer 202, the system performs multi-dimensional feature extraction and cross-agent interaction on the input information. Specifically, this layer includes a global guidance head, a spatial relationship head, a temporal dynamic head, and a resource contention head. The input data is first mapped and concatenated through a linear layer, and then input into the scaled dot product attention mechanism, where it is mapped to values ​​(V), keys (K), and queries (Q) respectively. Parallel computation is performed according to the dimension of each attention head to accurately capture the complex spatial and interactive relationships between different agents.

[0039] The extracted features are input into the feature fusion layer 203 for fusion and normalization. This layer specifically performs multi-head output concatenation, linear projection, residual connection, layer normalization, and random deactivation operations to enhance the model's generalization ability and prevent overfitting.

[0040] The processed features are input into decision network layer 204. The decision network is based on a multi-agent deep deterministic policy gradient (MADDPG) architecture, including an evaluation network and an execution network. The evaluation network is used to evaluate the value of actions during the centralized training phase, and its input is a multi-dimensional feature vector and local motion instructions, and its output is the evaluation value. The execution network is used to generate control instructions during the distributed execution phase, and its input is a multi-dimensional feature vector, and its output is local motion instructions.

[0041] Figure 3 The illustration shows an overall flowchart of a multi-agent navigation method according to an embodiment of this application.

[0042] like Figure 3 As shown, the steps are as follows: S301, acquire the state information and environmental information of each agent. Specifically, the state information includes the current position, velocity, attitude, and task status of each agent, and the environmental information includes static map data, dynamic obstacle distribution, and the real-time position of other agents.

[0043] S302 generates a global reference path for each agent based on state and environment information.

[0044] For example, in a smart warehousing scenario, a global reference path can plan paths suitable for the size and motion constraints of different types of automated guided vehicles, such as forklifts, lurking vehicles, and sorting vehicles.

[0045] Based on the starting point, target point, and static obstacle distribution in the environment of each agent, and combined with the physical parameters and motion constraints of each agent, a collision-free global reference path from the starting point to the target point is planned. This path consists of a series of ordered waypoints, providing long-term navigation guidance for the agents.

[0046] S303 generates a multi-dimensional feature vector based on environmental information and the generated global reference path. Specifically, multi-dimensional features are extracted and encoded from the environmental information and the global reference path to generate a multi-dimensional feature vector containing path guidance features, spatial obstacle features, temporal dynamic features, and resource competition features. These features respectively characterize the relationship between the agent and the global path, the distribution of surrounding obstacles, historical movement trends, and the occupancy status of key resource points, providing the input data foundation for the decision network.

[0047] S304, the multi-dimensional feature vector is input into the decision network. The decision network processes the multi-dimensional feature vector based on a multi-head attention mechanism, outputting local motion commands for each agent. Specifically, the encoded multi-dimensional feature vector is input into a pre-trained decision network. The decision network includes a global guidance head, a spatial relationship head, a temporal dynamic head, and a resource competition head, used to evaluate path intent conflicts, spatial collision risks, short-term interaction situations, and resource occupation order, respectively. By assigning differentiated weights to features of different dimensions and fusing them, the network ultimately calculates local motion commands containing information such as velocity and angular velocity, enabling collaborative decision-making among heterogeneous multi-agents and ensuring the coordination of actions among agents in complex environments.

[0048] S305 controls the movement of intelligent agents based on local motion commands. Specifically, the local motion commands output by the decision network are sent to each intelligent agent. Each intelligent agent adjusts its own motion state in real time according to the received local motion commands, and performs motion operations such as walking, turning, and speed adjustment. Based on the global reference path, it completes real-time dynamic adjustments to achieve real-time obstacle avoidance and path following during navigation.

[0049] Figure 4 The illustration shows a schematic diagram of the global reference path generation process of a multi-agent navigation method according to an embodiment of this application.

[0050] like Figure 4 As shown, the steps are as follows: S401, the obstacles in the environmental information are expanded based on the physical parameters of each agent. Specifically, considering the differences in physical parameters of each agent, physical parameters such as the radius of its circumscribed circle or the vehicle's outline are obtained. Before searching the path on the environmental map, obstacles in the map are dynamically expanded based on the physical parameters of each agent, with the expansion radius set as the sum of the agent's radius and a preset safety margin. Through this expansion process, agents of different sizes are uniformly abstracted into point masses, ensuring that the planned global reference path meets the collision-free passage requirements of the agent in geometric space.

[0051] (Formula 1) Formula 1 is the formula for calculating the dynamic expansion radius of an obstacle, where... This represents the radius of expansion of the obstacle set for the current agent. This represents the equivalent circumcircle radius or vehicle body contour radius of the current intelligent agent. This represents a pre-set safety distance constant. This formula can be used to adapt to intelligent agents of different sizes and ensure safe passage.

[0052] S402, based on the jump point search algorithm, generates a global reference path that conforms to the motion constraints of each agent after dilation. Specifically, based on the dilated map, the jump point search algorithm is used to search for a path from the agent's starting point to the target point to obtain the optimal path from the agent's starting position to the target position.

[0053] The jump point search algorithm selects key nodes in a jump-like manner, quickly generates an initial geometric path that avoids expanding obstacles, and combines the motion constraints of each agent, such as minimum turning radius or maximum curvature limit, to smooth the initial geometric path or optimize the trajectory, eliminate sharp turns in the path, and generate a global reference path that conforms to its motion constraints.

[0054] For example, for an agent with nonholonomic constraints, such as a forklift-type automated guided vehicle, the minimum turning radius is large, and the initial geometric path needs to be smoothed to meet its motion constraints.

[0055] Using this global reference path as a guiding benchmark for subsequent feature encoding and local decision-making ensures that the agent can make local dynamic adjustments based on global optimality.

[0056] The following is combined with Figure 5 The specific implementation process of the jump point search algorithm called in the aforementioned steps will be explained in detail.

[0057] Figure 5 The illustration shows a schematic diagram of the path generation process of a jump point search algorithm in a multi-agent navigation method according to an embodiment of this application.

[0058] like Figure 5 As shown, the steps are as follows: Step 501: Begin executing the jump point search algorithm path generation process.

[0059] Step 502, Load environmental information. Obtain environmental information including map data containing the distribution of static obstacles.

[0060] Step 503, obstacle expansion processing. Obstacles in the environmental information are expanded based on the physical parameters of each agent, such as size and motion constraints, to ensure that the subsequently generated path meets the safe passage requirements of the agents and avoids collisions.

[0061] Step 504: Construct the search space. A gridded search space is constructed based on the expanded environmental information.

[0062] Step 505: Initialize jump point search. Set the starting point and target point for each agent and construct a weighted heuristic function. The weighted heuristic function includes an actual path cost term, a distance heuristic term, and a task priority heuristic term.

[0063] Step 506, Neighbor Node Expansion. Starting from the current node, the search node is evaluated using a weighted heuristic function, and neighbor nodes are expanded along a straight line or diagonal. During this process, the value of the task priority heuristic term is set according to the task priority of each agent, so that agents with high task priority obtain lower heuristic values ​​during the search, thereby guiding the jump point search algorithm to prioritize the search towards the target point direction of agents with high task priority.

[0064] Step 507, Jump Point Identification. Based on the jump point search rules, determine if the currently expanding node is a jump point, i.e., a node with a forced neighbor. If not, continue to step 506 for expansion; if so, add it to the open list.

[0065] Step 508, Target Point Check. Determine if the current jump point is the target point. If not, return to step 506 to continue the search; if yes, proceed to the next step.

[0066] Step 509, Path backtracking optimization. Starting from the target point, backtrack along the parent node pointers to the starting point, generate the initial global path, and optimize it.

[0067] Step 510, Smoothing. The initial global path is smoothed, such as by removing redundant inflection points, to obtain a smoother path that better conforms to the agent's motion constraints.

[0068] Step 511: Output the global reference path. The smoothed path is output as the global reference path for each agent.

[0069] Step 512, end the process.

[0070] Figure 6 The illustration shows a schematic diagram of the multidimensional feature vector encoding process of a multi-agent navigation method according to an embodiment of this application.

[0071] like Figure 6 As shown, the steps are as follows: S601, based on waypoints located ahead of the agent's current position in the global reference path, generates path guidance features. Specifically, for each agent, its projection point on the global reference path at the current moment is obtained, and a preset number of waypoints to be traversed are selected forward. The relative position coordinates and orientation angles of each waypoint relative to the agent's own coordinate system are calculated, and after normalization, they constitute the path guidance features. This feature represents the path direction that the agent needs to follow over a certain distance in the future, providing long-term navigation intention information for the decision network and guiding the agent to move towards the target point along the global reference path.

[0072] (Formula 2) Formula 2 is the formula for characteristic normalization calculation, where This represents the normalized single-dimensional path guidance feature. This represents the original value of the waypoint's relative coordinates or heading angle. , These represent the minimum and maximum values ​​of the feature in this dimension, respectively. After normalization, the features are uniformly mapped to the [0,1] interval to ensure the stability of the input data.

[0073] S602, based on the distance and type information of obstacles surrounding the agent, spatial obstacle features are encoded and generated. Specifically, the 360-degree space around the agent is divided into multiple sector regions. Within each sector, the nearest obstacle is detected, its distance value is recorded, and the obstacle type is identified. The type information distinguishes static obstacles from other dynamic agents. For other detected dynamic agents, their relative velocity information is additionally encoded. The distance, type, and relative velocity information of each sector are combined to form the spatial obstacle features. This feature provides the decision network with real-time perception information of the space occupancy around the agent, used to assess the collision risk at the current moment.

[0074] S603, based on motion trend information extracted from the agent's historical state sequence, encodes and generates temporal dynamic features. Specifically, it acquires the agent's historical state information over multiple consecutive time steps, including temporal data such as position, velocity, and acceleration. The historical state sequence is processed through a recurrent neural network or a position encoding and attention mechanism to extract implicit motion trend information, such as velocity change trends and turning intentions, forming temporal dynamic features. This feature enables the decision network to predict the agent's own motion inertia and the short-term behavioral intentions of other surrounding agents.

[0075] S604, based on the occupancy status information and distance information of key resource points in the map, encodes and generates resource competition features.

[0076] For example, in a smart warehousing scenario, key resource points may include narrow aisles between shelves, intersections, and loading and unloading stations, and multiple automated guided vehicles need to coordinate their passage through these resource points.

[0077] Specifically, potential competing resource points in the environmental map are identified, including locations such as intersection centers, narrow passage entrances, and loading / unloading stations. For each resource point, the distance from the agent's current position to the resource point is calculated, and the current occupancy status of the resource point is obtained, including whether it is occupied by other agents, the identity of the occupant, the estimated release time, and the number of agents waiting in the queue. This information is encoded as resource competition features to characterize the spatial relationship and competitive situation between agents and key resource points.

[0078] S605 integrates path guidance features, spatial obstacle features, temporal dynamic features, and resource competition features to generate a multi-dimensional feature vector. Specifically, the generated path guidance features, spatial obstacle features, temporal dynamic features, and resource competition features are each mapped to a unified feature dimension through a fully connected layer. Then, the mapped feature vectors are concatenated along the feature dimension to form a multi-dimensional feature vector. This multi-dimensional feature vector contains information from multiple dimensions, including global path guidance, local spatial awareness, temporal motion trends, and resource competition dynamics. It serves as input data for the decision network, providing an environmental state representation for subsequent local motion command generation.

[0079] Figure 7 The illustration shows a schematic diagram of the local motion command generation process of a multi-agent navigation method according to an embodiment of this application.

[0080] like Figure 7 As shown, the steps are as follows: S701, inputting the multidimensional feature vector into the decision network. Specifically, the generated multidimensional feature vector is used as input data and fed into the pre-trained decision network. This multidimensional feature vector integrates path guidance features, spatial obstacle features, temporal dynamic features, and resource competition features, containing multidimensional information such as global path direction, local obstacle distribution, movement trend prediction, and resource competition situation, providing a complete environmental state representation for the decision network to make collaborative decisions.

[0081] S702, through the global guidance head in the decision network, evaluates the path intent conflicts between agents based on the global reference path. Specifically, the global guidance head in the decision network uses the current agent's path guidance features as queries and the path guidance features of other agents as keys and values, and calculates the correlation between the current agent and other agents in global path intent through an attention mechanism. When the global reference paths of multiple agents show a tendency to intersect or overlap in future spatiotemporal events, the global guidance head outputs a higher attention weight, representing the potential long-term path conflict risk and providing information for subsequent cooperative obstacle avoidance.

[0082] S703, through the spatial relationship head in the decision network, assesses the instantaneous spatial collision risk based on the distance information of obstacles around each agent. Specifically, the spatial relationship head in the decision network uses the spatial obstacle features of the current agent as the query and the spatial obstacle features of other agents as the key and value, and calculates the spatial correlation between the current agent and neighboring agents and obstacles through an attention mechanism. This spatial relationship head determines the danger level of each interactive object based on distance and relative velocity information, and outputs the assessment result of the instantaneous spatial collision risk to guide the agent's emergency obstacle avoidance behavior at the current moment.

[0083] S704 predicts short-term interaction dynamics based on the historical state sequence information of each agent through the temporal dynamic head in the decision network. Specifically, the temporal dynamic head in the decision network uses the temporal dynamic features of the current agent as the query and the temporal dynamic features of other agents as keys and values, and analyzes the movement trends in the historical motion trajectories of each agent through an attention mechanism. By determining whether other agents are in a state of acceleration, deceleration, turning, or constant speed, this temporal dynamic head predicts the interaction dynamics in the short term, enabling agents to make coordinated responses in advance and avoid sudden conflicts.

[0084] S705 coordinates the order of spatial resource occupancy based on information about key resource points on the map by using resource competition heads in the decision network.

[0085] For example, in an intersection scenario, the resource competition head can coordinate the orderly passage of automated guided vehicles from different directions, avoiding deadlocks or congestion caused by competition for right-of-way.

[0086] Specifically, the resource contention head in the decision network uses the resource contention characteristics of the current agent as the query and the resource contention characteristics of other agents as keys and values. It then uses an attention mechanism to evaluate the competitive situation of multiple agents regarding the same key resource point, such as an intersection or a narrow passage. This resource contention head comprehensively considers the distance each agent travels to the resource point, task priority, and current occupancy status to generate attention weights for resource contention. This guides the agents to form an orderly passage sequence, avoiding deadlocks or congestion caused by resource contention.

[0087] S706 integrates the outputs of the global guidance head, spatial relationship head, temporal dynamic head, and resource contention head to generate local motion commands for each agent. Specifically, the decision network weights and fuses the outputs of the global guidance head, spatial relationship head, temporal dynamic head, and resource contention head. An attention mechanism assigns differentiated weights to the outputs of different heads, ultimately decoding and generating local motion commands for each agent. These local motion commands include linear velocity and angular velocity, used to control the agent's real-time movement and ensure that the agent can make safe, efficient, and coordinated motion decisions in complex dynamic environments.

[0088] (Formula 3) Formula 3 is the weighted fusion formula for multi-head output features, where This represents the features after weighted fusion. , , , These are the output features of the global guidance header, spatial relationship header, temporal dynamic header, and resource contention header, respectively. ~ These are the weight coefficients for adaptive learning, which, after fusion, are used to generate the corresponding local motion commands for each agent.

[0089] Figure 8 The illustration shows a schematic diagram of the hierarchical reward function evaluation process of a multi-agent navigation method according to an embodiment of this application.

[0090] like Figure 8 As shown, the steps are as follows: S801, Obtain the state information of each agent after executing the local motion command. Specifically, after the decision network outputs the local motion command and controls the agents to execute it, obtain the state information of each agent at the next moment, including the updated position, velocity, attitude, and distance to the target point. The above state information reflects the actual motion effect of the agent after executing the local motion command, providing a data basis for subsequent reward calculation.

[0091] S802 calculates the reward value for each agent based on a hierarchical reward function. This hierarchical reward function includes path guidance reward, safety obstacle avoidance reward, cooperative interaction reward, and task progress reward, which are used to guide the agent to follow the global reference path, avoid surrounding obstacles, cooperate to pass key resource points, and move towards the target point, respectively.

[0092] Specifically, the reward value for each layer of reward items is calculated based on the state information of the agent after executing local motion instructions.

[0093] Path guidance rewards are evaluated based on the degree of deviation of the agent's current position from the global reference path. The agent receives a positive reward when it travels along the global reference path and is penalized when it deviates from the path.

[0094] The safety obstacle avoidance reward is evaluated based on the distance between the agent and surrounding obstacles and other agents. The closer the distance, the greater the penalty, in order to encourage the agent to maintain a safe distance.

[0095] The collaborative interaction reward evaluates the passage of key resource points. When multiple agents pass through intersections or narrow passages in an orderly and efficient manner, they receive positive rewards, while they are penalized when congestion or deadlock occurs.

[0096] Task progress rewards are evaluated based on the reduction in distance between the agent and the target point; each step forward towards the target point earns a positive incentive. All rewards are weighted and summed according to preset weights to obtain a comprehensive reward value, which is used to quantify the agent's performance at the current moment.

[0097] (Formula 4) Formula 4 is the comprehensive calculation formula for tiered rewards, where This represents the overall reward value. , , , These are the calculation results for path guidance rewards, safety obstacle avoidance rewards, collaborative interaction rewards, and task progress rewards, respectively. ~ Preset weights are used to quantify and evaluate the navigation behavior of intelligent agents and guide network training.

[0098] S803 updates the parameters of the decision network based on the reward value.

[0099] Specifically, during training, the calculated comprehensive reward value is used as a feedback signal to iteratively update the parameters of the decision network using a policy gradient algorithm or a temporal difference learning algorithm. By maximizing the comprehensive reward value as the optimization objective, the decision network is guided to learn a better cooperative strategy, enabling the output local motion commands to better balance global path following, local obstacle avoidance safety, multi-agent cooperation, and task completion efficiency.

[0100] Figure 9 The illustration shows a schematic diagram of the global reference path replanning process of a multi-agent navigation method according to an embodiment of this application.

[0101] like Figure 9 As shown, the steps are as follows: S901 monitors changes in environmental information and / or the path intent conflict assessment results output by the global guidance head. Specifically, during the agent's movement, environmental information is continuously monitored in real time to identify environmental changes such as the addition or removal of static obstacles and the sudden intrusion of dynamic obstacles. Simultaneously, the attention weights and path intent conflict assessment results output by the global guidance head in the decision network are acquired in real time. These results reflect the potential intersection or overlap risks between the global reference paths of each agent. Continuous monitoring of the above information provides a data basis for triggering global reference path replanning.

[0102] S902, when a change in environmental information is detected and / or a path intent conflict is predicted between agents, the affected agents are identified.

[0103] For example, when a temporary obstacle such as newly stacked goods suddenly appears in the warehouse, the affected automated guided vehicles need to replan their routes to avoid the area.

[0104] Specifically, when a significant change in the distribution of obstacles in the environmental map is detected, resulting in a collision risk on the original global reference path, or when the output of the global guidance head indicates that the global reference paths of multiple agents have a tendency to intersect or overlap in future spatiotemporal events, and the conflict risk exceeds a preset threshold, a replanning condition is triggered. Based on this, the scope of affected agents is further identified, including agents whose paths are impassable due to environmental changes, and agents whose path intentions conflict with those of other agents, thus defining the targets for subsequent replanning.

[0105] S903, Regenerate Global Reference Paths for the Affected Agents. Specifically, for the identified affected agents, based on the latest environmental information and the physical parameters and motion constraints of each affected agent, perform obstacle expansion again, and plan new global reference paths for each agent based on the jump point search algorithm. The newly generated global reference paths will replace the original global reference paths and serve as the guiding benchmark for subsequent feature encoding and local decision-making, ensuring that the agents can always travel towards the target point along a collision-free path that conforms to motion constraints in a dynamically changing environment.

[0106] Figure 10 The illustration shows a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application.

[0107] like Figure 10As shown, the electronic device 1000 includes a processor 1001 and a memory 1002. The memory 1002 stores a computer program, which, when executed by the processor 1001, enables the implementation of any of the aforementioned multi-agent navigation methods. The processor 1001 may be a central processing unit, a graphics processing unit, a neural network processor, or other general-purpose or special-purpose processing chip, and the memory 1002 may be a read-only memory, random access memory, flash memory, or hard disk, among other storage media. This electronic device can be deployed in a cloud server, an edge computing node, or the vehicle-mounted computing unit of an intelligent agent, serving as the physical execution carrier of the navigation method of this application.

[0108] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0109] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” and “having” are open-ended terms meaning “including but not limited to” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to” and is used interchangeably with it.

[0110] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0111] It should also be noted that in the system and method of this application, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of this application.

[0112] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this application is not limited to the specific aspects of the processes, machines, manufacturing, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufacturing, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described above can be utilized. Therefore, the appended claims include such processes, machines, manufacturing, events, means, methods, or actions within their scope.

[0113] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0114] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A multi-agent navigation method, characterized in that, include: Acquire the state and environmental information of each agent; Based on the state information, the environment information, and the physical parameters of each agent, a global reference path that conforms to the motion constraints is generated for each agent. Based on the environmental information and the global reference path, a multi-dimensional feature vector is generated; The multidimensional feature vector is input into the decision network; wherein, the decision network uses a multi-head attention mechanism to perform attention operations on the path guidance features, spatial obstacle features, temporal dynamic features and resource competition features in the multidimensional feature vector respectively, and the results are weighted and fused to obtain the local motion instructions of each agent; The agent is controlled to move according to the local motion instructions.

2. The multi-agent navigation method according to claim 1, characterized in that, Generating a global reference path for each of the aforementioned agents that conforms to their motion constraints includes: The obstacles in the environmental information are expanded according to the physical parameters of each of the intelligent agents; Based on the jump point search algorithm, a global reference path that conforms to the motion constraints of each agent after the dilation process is generated.

3. The multi-agent navigation method according to claim 1, characterized in that, Generating the multidimensional feature vector includes: Based on waypoints located ahead of the current position of each agent in the global reference path, the relative coordinates and orientation angles of each waypoint relative to the agent are calculated and normalized to generate the path guidance features; The space surrounding each of the intelligent agents is divided into multiple sectors. Within each sector, the type of the nearest obstacle, as well as its distance and relative speed with respect to the intelligent agent, are detected to generate the spatial obstacle features. The historical state sequence of each agent is encoded using a recurrent neural network to extract motion trend information and generate the temporal dynamic features. Identify key resource points in the map, encode the distance from each agent to the resource point, the resource point's occupancy status, and waiting queue information, and generate the resource contention features.

4. The multi-agent navigation method according to claim 1, characterized in that, Inputting the multidimensional feature vector into the decision network includes: Extract the path guidance features, spatial obstacle features, temporal dynamic features, and resource competition features from the multidimensional feature vector; The path guidance features are input into the global guidance head in the decision network to evaluate the path intent conflicts between the agents and generate a first evaluation result. The spatial obstacle features are input into the spatial relationship head of the decision network to assess the instantaneous spatial collision risk between the current agent and other agents and obstacles in the surrounding area, and generate a second assessment result. The time-series dynamic features are input into the time-series dynamic head of the decision network to predict the short-term interaction situation between the agents and generate a third evaluation result. The resource competition characteristics are input into the resource competition head in the decision network to schedule the order of spatial resource occupation by each agent and generate a fourth evaluation result. The first evaluation result, the second evaluation result, the third evaluation result, and the fourth evaluation result are weighted and fused to generate the local motion command.

5. The multi-agent navigation method according to claim 1, characterized in that, During the training of the decision network, the behavior of the agent is evaluated based on a hierarchical reward function; The hierarchical reward function includes: Path guidance reward is calculated based on the degree of deviation between the agent's current position and the global reference path, in order to guide the agent to follow the global reference path; The safety obstacle avoidance reward is calculated based on the distance between the agent and surrounding obstacles, and a risk penalty coefficient is set according to the physical parameters of each agent to guide the agent to maintain a safe distance. The collaborative interaction reward is calculated based on the time taken and task value weights of multiple agents through key resource points, in order to guide multiple agents to collaborate through key resource points. The task progress reward is calculated by normalizing the ratio of the distance the agent travels to the target point to its maximum speed, in order to guide the agent to move towards the target point.

6. The multi-agent navigation method according to claim 4, characterized in that, The method further includes dynamically replanning the global reference path, specifically including: During the movement of the intelligent agent, the first evaluation result output by the global guidance head is acquired in real time, and changes in the environmental information are monitored. When a change in the environmental information is detected, and / or when a path intent conflict is predicted between the agents based on the first evaluation result and the risk exceeds a preset threshold, the affected agents whose paths are impassable or whose path intent conflicts exist due to environmental changes are identified, and a new global reference path conforming to their motion constraints is generated for the affected agents. The path-guided features in the multidimensional feature vector are updated based on the regenerated global reference path.

7. The multi-agent navigation method according to claim 4, characterized in that, The resource contention head schedules the order in which the agents occupy spatial resources, including: Use the resource contention characteristics of the current intelligent agent as the query, and use the resource contention characteristics of other intelligent agents as the key and value; Based on the distance of each agent to the key resource point, the task priority, and the current occupancy status of the resource point, the attention weight is calculated. The agents are scheduled to form an orderly passage order based on the attention weights.

8. The multi-agent navigation method according to claim 2, characterized in that, The jump point search algorithm generates a global reference path that conforms to the motion constraints of each agent after dilation processing, including: In the path search process of the jump point search algorithm, a weighted heuristic function is used to evaluate the search nodes; wherein, the weighted heuristic function includes an actual path cost term, a distance heuristic term, and a task priority heuristic term; The value of the task priority heuristic is set according to the task priority of each agent, so that agents with high task priority obtain lower heuristic values ​​during the search, thereby guiding the jump point search algorithm to prioritize the search towards the target point of the agent with high task priority.

9. The multi-agent navigation method according to claim 1, characterized in that, The decision network includes an execution network and an evaluation network, and the training process of the decision network includes: The decision network is trained using a centralized training mechanism, and the evaluation network obtains multi-dimensional feature vectors and local motion instructions from all agents. The evaluation network uses a multi-head attention mechanism to process and weight the multi-dimensional feature vectors of all agents, evaluates the fused features with the local motion commands of all agents, and outputs an evaluation value to update the parameters of the execution network.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the multi-agent navigation method according to any one of claims 1 to 9.

Images