Multi-robot path planning method in complex environment based on MQAHA

Abstract

The invention discloses a multi-robot path planning method in a complex environment based on MQAHA, and belongs to the technical field of multi-robot path planning. According to the method, firstly, an optimal point set initialization strategy based on the number theory is adopted, an initial population which is evenly distributed is constructed, and diversity and coverage of a search starting point are remarkably improved; secondly, proposing an elite-guided variation migration foraging strategy, fusing direction guidance of a current optimal solution and a disturbance mechanism of a random individual, and effectively enhancing local development capability and global escape performance; furthermore, a learning-driven strategy self-adaptive selection mechanism is introduced, the foraging behavior probability is dynamically adjusted according to the population state, efficient balance of exploration and development is achieved, and therefore the premature convergence problem of the algorithm is systematically relieved. According to the method, the multi-robot cooperative motion path can be quickly generated in a complex obstacle environment, static and dynamic obstacles are effectively avoided while the total path length is reduced, and the motion efficiency and operation safety of a multi-robot system are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-robot path planning, and particularly relates to a multi-robot path planning method in a complex environment based on MQAHA. BACKGROUND

[0002] In recent years, with the rapid development of robot technology, mobile robots have gradually expanded from traditional industrial production lines to diversified application scenarios such as intelligent logistics, modern agriculture, search and rescue, and environmental survey. In these tasks, a single robot is often limited by its perception ability, work efficiency and load capacity, and it is difficult to complete complex tasks independently. In contrast, multi-robot systems, through information sharing and collaborative work, exhibit higher robustness, flexibility and overall efficiency, and have been widely applied in intelligent warehouses, intelligent manufacturing factories, oil field inspection and other fields.

[0003] Multi-robot path planning (MRPP) is a core technology for realizing autonomous collaboration of multi-mobile robot systems. The core goal is to plan a safe, collision-free and efficient motion path for each robot in the system from the starting point to the end point in a complex environment containing static or dynamic obstacles. For a single robot, path planning is essentially a multi-objective optimization problem (MOOP), which needs to consider path length, safety and real-time performance; while in the multi-robot scenario, additional collision avoidance constraints and dynamic interaction between robots need to be considered, making problem modeling and solving more complex.

[0004] Traditional path planning algorithms, such as Dijkstra's algorithm, A* algorithm and artificial potential field (APF), can effectively solve the single robot path planning problem. However, when applied to complex, dynamic and high-dimensional multi-mobile robot systems, the limitations of these traditional algorithms become apparent. For example, Dijkstra's algorithm can guarantee to find the optimal path, but its search range is large, the computational complexity is high, and it is inefficient in handling large-scale maps, making it difficult to meet real-time requirements. A* algorithm faces difficulties in designing heuristic functions in dynamic environments and lacks mechanisms to efficiently solve dynamic conflicts between robots. APF, although simple to calculate and real-time, has the fatal flaw of being easily trapped in local minimum points, which can easily lead to robots failing to reach the target and failing to plan an effective path.

[0005] To overcome the above limitations, researchers' focus has systematically shifted to metaheuristic optimization algorithms. Such algorithms are inspired by the biological evolution or swarm intelligence behavior in nature, and have the advantages of flexible structure, low dependence on problem models, and strong adaptability. Typical representatives such as Particle Swarm Optimization (PSO), Genetic Algorithm, and Ant Colony Optimization have achieved certain results in mobile robot path planning. In recent years, a new metaheuristic algorithm, Artificial Hummingbird Algorithm (AHA), has received widespread attention due to its unique mechanism. AHA simulates three foraging behaviors of hummingbirds (guided foraging, territorial foraging, and migratory foraging) and their axial, diagonal, and omnidirectional flight patterns, achieving a good balance between exploration and development. Studies have shown that AHA has faster convergence speed and higher optimization accuracy than traditional algorithms on a variety of benchmark optimization problems. Currently, AHA has been successfully applied to multiple engineering fields, such as the design of a hybrid optimization framework AHA-MPA that combines AHA and Marine Predators Algorithm (MPA) for the primary frequency regulation of a microgrid PI-PIDA-driven STATCOM in the paper "Imtiaz S, Yang L, Munir H M, et al. Hybrid metaheuristic algorithm-based proportional-integral and proportional-integral-derivative-acceleration combination controller (PI-PIDA) driven static synchronous compensator (STATCOM) for optimizing microgrid frequency stability [J]. Energy Reports, 2025, 13: 2907-293". The results show that the AHA-MPA hybrid algorithm significantly reduces the frequency overshoot, achieves the best steady-state frequency recovery, and provides the maximum reactive power support in both transient and steady states, proving the effectiveness and engineering applicability of the improved AHA in microgrid frequency stability control. In addition, AHA has also been applied to voice signal-based Parkinson's disease feature selection, waste classification and feature selection problems, task scheduling, and prediction of composite material wear. Therefore, AHA and its variants have great potential in solving multi-mobile robot path planning in dynamic and complex environments.

[0006] Although AHA performs well on standard optimization problems, there are still obvious limitations in dealing with complex optimization tasks. One of them is that the population diversity is prone to be lost prematurely, because the individual updating mechanism is relatively fixed, which makes the diversity decline rapidly during the search process, eventually leading the algorithm to fall into a local optimal solution. At the same time, the adaptive ability of the algorithm is also insufficient, and the strategy selection presents a static feature, which cannot dynamically adjust the search behavior according to the iteration stage or problem characteristics, resulting in unstable convergence process and significant fitness value fluctuations. In addition, the standard AHA also has the problem of imbalance between exploration and development, especially in high-dimensional complex search space, its global exploration ability is insufficient, and it is difficult to effectively cover a wide range of feasible solution regions, which restricts the potential to solve complex problems. According to the "no free lunch" theorem, there is no universal optimal algorithm suitable for all optimization problems. Therefore, when AHA is directly applied to the MRPP scene, its inherent limitations (such as insufficient search accuracy, frequent path conflicts, and high risk of deadlock) are highlighted, making it difficult to guarantee the safety, optimality, and real-time performance of the path.

[0007] In summary, low path quality, real-time performance, and collaborative efficiency are the core bottlenecks of existing algorithms in dealing with complex MRPP scenarios, directly restricting the stability and practicality of multi-robot systems. Therefore, it is urgent to propose a new path planning method that combines strong exploration ability, adaptive mechanism, and efficient collaboration strategy to cope with the MRPP challenges in dynamic complex environments. SUMMARY

[0008] The technical problem to be solved by the present application is to provide a multi-robot path planning method in complex environments based on MQAHA (Multi-Strategy Q-learning Artificial Hummingbird Algorithm, MQAHA), aiming to efficiently obtain a globally optimal or near-optimal path planning scheme in complex environments. The specific performance is to simultaneously optimize the path length and obstacle avoidance ability at each planning step, thereby significantly shortening the total path, avoiding collisions between robots and obstacles, and ultimately improving the overall safety and efficiency of the multi-robot system operation.

[0009] To solve the technical problems of the present application, the technical scheme adopted by the present application is as follows: a multi-robot path planning method in complex environments based on MQAHA, which comprises the following steps:

[0010] Step 1, construct a robot motion model, assuming that all robots are uniform circular rigid bodies, and their positions at time t are calculated by the following formula (15):

[0011] (15) ​

[0012] wherein, is the position coordinate of the i-th robot at time t, is the current position coordinate of the i-th robot, is the velocity and radial angle of the i-th robot, respectively, is the time for the i-th robot to move at velocity is the position coordinate of the i-th robot at time t, is the time for the i-th robot to move at velocity

[0013] Step 2, based on the above motion model, a comprehensive evaluation function is constructed, which is used to comprehensively optimize the path length, obstacle avoidance ability and anti-collision performance between robots in the process of multi-robot cooperative motion. The evaluation function adopts the weighted summation method to construct the total fitness function, and its calculation formula is as follows:

[0014] (21)

[0015] wherein, is the fitness function of the shortest distance of the robot to reach the target point, is the fitness function of avoiding collision with obstacles, is the fitness function of avoiding collision with other robots;

[0016] Step 3, initialize the current position list of all mobile robots and its corresponding target position list;

[0017] Step 4, judge whether all robots have reached their respective target points, if all arrive, the path planning is ended; otherwise, enter step 5;

[0018] Step 5, determine the next motion state of each robot by using MQAHA and update its current position, then return to step 4, wherein MQAHA is based on the velocity and radial angle of all robots, that is, the input variables are: wherein and the search space ranges are and respectively, the robot will move to the feasible position in turn until it reaches the target point, and MQAHA will learn and combine with AHA algorithm, giving AHA self-learning ability; by comprehensively considering the historical state, current state and future state of the population, the self-adaptive selection of foraging strategy is realized, so as to improve the robustness and efficiency of path planning.

[0019] Fitness function of the shortest distance of the robot to reach the target point is expressed as:

[0020] ​​​ (16)

[0021] where, is the position coordinate of the i-th robot at time t, is the target position coordinate of the i-th robot; is the position coordinate of the i-th robot at time t,

[0022] Fitness function for avoiding collision with obstacles is expressed as:

[0023] (17)

[0024] where, is the penalty value, is the safety collision distance, is the sum of distances between robots and obstacles, which is specifically calculated as follows:

[0025] (18)

[0026] where, and are the number of robots and obstacles, respectively, is the position of the i-th obstacle, is the position of the i-th robot. Fitness function for avoiding collision with other robots is expressed as:

[0027]

[0028] (19)

[0029] where, and have the same meaning as in function is the sum of distances between robots, which is specifically calculated as follows:

[0030] (20)

[0031] where, is the position of the i-th robot, and , is the position of the i-th robot.

[0032] The implementation steps of MQAHA are as follows:

[0033] Step 5.1, generate initial candidate solution:​​​​​​​ ;

[0034] Step 5.2: Evaluate the objective function value using formula (21);

[0035] Step 5.3, according to The value table adjusts the probabilities of guided foraging and territorial foraging, and then selects guided foraging or territorial foraging based on the foraging probability;

[0036] Step 5.4: Guide the selected foraging behavior to engage in corresponding foraging or territorial foraging.

[0037] Step 5.5: Update the population and population based on the results of Step 5.4. Value table;

[0038] Step 5.6: Determine whether the MQAHA termination condition is met. If it is met, update the current position list and return to step 4; otherwise, return to step 5.3.

[0039] MQAHA uses the optimal point set method when initializing the population. The specific steps for initializing the optimal point set are as follows:

[0040] Let the population size be The decision space dimension is Then the formula for generating the optimal point set is:

[0041] (7)

[0042] In the formula, For the best point, take , , To meet The smallest prime number of the population is given by formula (7). The population generation formula is as follows:

[0043] (8)

[0044] In the formula, For the first Individual in the first The position of the dimension For the best point of concentration Individual in the first Dimension value, and The first The lower and upper limits of values ​​for each dimension.

[0045] The elite-guided mutation migration foraging method in step 5.4 is as follows:

[0046] Introducing mutation behavior guided by elite individuals, the mutation vector is first generated:

[0047] (9)

[0048] In the formula, For migrating individuals, Scaling factor The globally optimal elite individual For each random individual in the population, binomial mutation is then performed:

[0049] (10)

[0050] In the formula, The variability rate, The first of the variant vectors The dimension value, rand, is a random number between 0 and 1. By introducing elite individuals, the migrating individuals are made to move closer to the elite individuals, while random individuals and mutation behavior are introduced to generate random perturbations.

[0051] In step 5.5 The value table will be updated using formula (11):

[0052] (11)

[0053] In the formula, In the state Next action Corresponding value; The learning rate is used to adjust the relative weights of historical experience and immediate rewards to the current estimate. The discount rate measures the degree to which future earnings diminish relative to immediate earnings. For state Next action The rewards received.

[0054] Compared with the prior art, the present invention has the following significant advantages:

[0055] 1) Improved Population Initialization Mechanism: To address the issues of uneven population distribution and slow convergence caused by random initialization in the original AHA algorithm, a best-point-set initialization method is introduced. This method can generate an initial solution set with a more uniform spatial distribution and greater diversity, significantly improving the algorithm's convergence speed and accuracy, thereby enabling the planning of shorter and smoother paths for multi-robot systems in complex environments.

[0056] 2) Enhancing the information utilization efficiency of the migration foraging strategy: To address the problem that the original algorithm relies solely on random perturbations to generate new solutions during the migration foraging phase and fails to adequately utilize historical population information, an elite-guided mutation-based migration foraging strategy is proposed. This strategy utilizes the current optimal direction to guide the search, thereby strengthening the algorithm's local exploitation capabilities.

[0057] 3) Adaptive selection of foraging strategies: The original AHA algorithm uses a fixed probability to select foraging strategies, lacking dynamic adjustment capabilities. Therefore, an adaptive selection strategy based on... An adaptive foraging strategy selection mechanism based on learning. This mechanism dynamically adjusts the selection probability of guided foraging and territorial foraging by evaluating the historical and current states of the population in real time, achieving a dynamic balance between exploration and development. This not only effectively reduces the collision risk in multi-robot systems but also significantly improves obstacle avoidance capabilities and collaborative efficiency in dynamic obstacle environments.

[0058] 4) Comprehensive verification of algorithm performance and practicality: Comparative experiments were conducted on the standard CEC-2017 test set with six mainstream metaheuristic algorithms, including Dung Beetle Optimizer (DBO), Grey Wolf Optimizer (GWO), Whale Optimization Algorithm (WOA), Beluga Whale Optimization (BWO), Harris Hawks Optimization (HHO), and PSO. The results show that MQAHA has significant advantages in convergence speed, solution accuracy, and stability. Furthermore, simulation experiments of multi-robot online path planning were carried out in two dynamic fields with different complexities, which fully verified the effectiveness and robustness of the invention in practical applications.

[0059] In summary, the multi-robot path planning method based on MQAHA proposed in this invention can efficiently search for global near-optimal solutions in complex and dynamic environments, and simultaneously optimize path length, obstacle avoidance safety, and multi-robot cooperation in each decision step, significantly improving the operating efficiency and task reliability of multi-robot systems. Attached Figure Description

[0060] Figure 1 A comparison chart of initial population generation methods;

[0061] Figure 2 for Learning-driven AHA framework diagram;

[0062] Figure 3 MQAHA flowchart;

[0063] Figure 4 Convergence curve of example function in CEC2017;

[0064] Figure 5 MQAHA multi-mobile robot path planning flowchart;

[0065] Figure 6 A scenario diagram for multi-robot path planning;

[0066] Figure 7 A schematic diagram of the path planning for scenario 1;

[0067] Figure 8 A schematic diagram of the planned path for scenario 2. Detailed Implementation

[0068] Example 1: As Figures 1-8 As shown, a multi-robot path planning method based on MQAHA in complex environments is presented. This method includes the following steps:

[0069] Step 1: Construct a robot motion model:

[0070] Online path planning for multiple mobile robots is essentially an optimization problem. Its goal is to guide each robot safely and efficiently to its respective target point in a complex environment with minimal overall cost, while effectively avoiding static or dynamic obstacles and collisions with other robots. To achieve this goal, this invention constructs a multi-robot cooperative motion scenario, in which all robots are modeled as uniformly sized, circular rigid bodies. The robots... The position of the engraving is calculated using the following formula (15):

[0071] (15)

[0072] In the formula, For the first A robot Position coordinates at that moment Its current position coordinates, and These are the robot's velocity and radial angle, respectively. For the first A robot at speed The duration of the next movement. To determine the time of each robot's movement. At any given moment, the system will perform an optimization once per increment of time. This optimization is based on the velocities and radial angles of all robots; the input variables are: in and The scope of the search space is as follows and The robots will move sequentially to these feasible locations until they reach their respective target points. However, the movement of multiple mobile robots also faces the problems of planning the shortest path, obstacle avoidance, and preventing collisions between robots. Therefore, it is necessary to design corresponding evaluation functions, as shown in step 2.

[0073] Step 2: Construct an evaluation function based on the robot's movement model. This evaluation function comprehensively measures the overall performance of path planning for multiple mobile robots during movement, specifically including three core objectives: ① minimizing path length; ② effectively avoiding static or dynamic obstacles in the environment; ③ avoiding collisions with other robots. To achieve multi-objective collaborative optimization, this invention models the above objectives as corresponding fitness functions and constructs the overall fitness function through a weighted summation method. The calculation formula is shown below:

[0074] (twenty one)

[0075] In the formula, Let be the fitness function that minimizes the distance the robot takes to reach the target point. The fitness function to avoid collisions with obstacles, A fitness function to avoid collisions with other robots.

[0076] The fitness function for the shortest distance the robot takes to reach the target point. Represented as:

[0077] (16)

[0078] In the formula, For the first A robot Position coordinates at that moment For the first The target position coordinates of the robot;

[0079] Fitness function for avoiding collisions with obstacles Represented as:

[0080] (17)

[0081] In the formula, The penalty value. For safe collision distance, The sum of the distances between the robot and the obstacles is calculated using the following formula:

[0082] (18)

[0083] In the formula, and The numbers of robots and obstacles are respectively. For the first The location of the obstacle For the first The location of the robot.

[0084] Fitness function to avoid collisions with other robots Represented as:

[0085] (19)

[0086] In the formula, and AND function The meanings are exactly the same. The sum of the distances between the robots is calculated using the following formula:

[0087] (20)

[0088] In the formula, For the first The location of the robot, and , For the first The location of the robot.

[0089] Step 3: Initialize the current location list and the corresponding target location list of all mobile robots;

[0090] Step 4: Determine if all robots have reached their respective target points. If all have, path planning ends; otherwise, proceed to Step 5.

[0091] Step 5: Use MQAHA to determine the next motion state of each robot and update its current position, then return to step 4. MQAHA is based on the velocity and radial angle of all robots; the input variables are: ,in and The scope of the search space is as follows and The robots will move sequentially to feasible locations until they reach their respective target points. MQAHA will... By combining learning with the AHA algorithm, AHA is endowed with self-learning capabilities. By comprehensively considering the historical, current, and future states of the population, adaptive selection of foraging strategies is achieved, thereby improving the robustness and efficiency of path planning.

[0092] The traditional AHA algorithm is a metaheuristic swarm intelligence optimization algorithm based on the foraging behavior of hummingbirds. In nature, hummingbirds can rely on memory to flexibly choose foraging strategies and flight patterns among different food sources, thus efficiently locating high-yield food sources. AHA mathematically simulates this intelligent foraging process, guiding individual hummingbirds to gradually approach the global optimum in the solution space. Its main components are as follows:

[0093] The access table is a key data structure in AHA, used to record the access history of each hummingbird in the population to various candidate solutions (food sources). The value at each position in the table reflects the length of time that food source has not been visited by that hummingbird. The larger the value, the longer the food source has been "idle," and the algorithm assumes that it has accumulated more nectar (i.e., has higher exploration value). This mechanism effectively guides hummingbirds to prioritize exploring long-unvisited areas, thus significantly enhancing the algorithm's global exploration capability and helping to avoid prematurely getting trapped in local optima.

[0094] Guided foraging: Guided foraging is one of the core mechanisms of AHA (Active Harmonizing Exploration). In this phase, the hummingbird selects target food sources based on the "unvisited time" recorded in its access log, thus promoting global exploration. The mathematical model for this behavior is as follows:

[0095] (1)

[0096] In the formula, In order to be in The newly generated solution at each moment (i.e., the new position of the hummingbird). For the first A food source in Location at any given moment For the first The location of the target food source chosen by the hummingbird. The specific form of flight skill depends on the hummingbird's flight pattern (axial, diagonal, or omnidirectional flight). As a guiding factor, it follows a standard normal distribution. The guidance.

[0097] Territory foraging: Territory foraging is the local development phase of AHA, simulating the behavior of a hummingbird performing a refined search in the vicinity of its current location. This phase helps improve the local search accuracy of the algorithm by introducing random perturbations within the neighborhood of the current solution. Its mathematical model is as follows:

[0098] (2)

[0099] In the formula, The neighborhood factor follows a standard normal distribution. It is used to control the range of random perturbations in the local search.

[0100] Migratory foraging: Migratory foraging is a key mechanism in AHA that enhances global exploration capabilities. When the food source (i.e., candidate solution) in a certain area consistently yields poor returns, hummingbirds will migrate long distances to find new, potentially high-quality areas. In the algorithm implementation, this behavior is performed by the individual with the worst fitness value (i.e., the one whose food source has the worst nectar replenishment rate). The mathematical model is as follows:

[0101] (3)

[0102] In the formula, It is the food source with the worst nectar replenishment rate in the population. The migration coefficient follows a standard normal distribution. , and These are the upper and lower bounds of the optimization variables, respectively.

[0103] The flight skills that hummingbirds use during the foraging phase are generated by the hummingbird's flight patterns (axial, diagonal, or omnidirectional flight).

[0104] Axial flight: Axial flight simulates the behavior of a hummingbird flying along any coordinate axis in the search space. Its mathematical model is as follows:

[0105] (4)

[0106] In the formula, Let be the dimension of the solution vector.

[0107] Diagonal flight: Diagonal flight simulates the movement of a hummingbird along the diagonal direction of the search space. Its mathematical model is as follows:

[0108] (5)

[0109] In the formula, For a number from 1 to A random permutation (without duplicates). Let be a random variable that is uniformly distributed in the interval [0,1].

[0110] Omnidirectional flight: Omnidirectional flight simulates the ability of a hummingbird to move freely in any direction along the projections of each coordinate axis within the search space. Its mathematical model is as follows:

[0111] (6)

[0112] This invention proposes a method that integrates multiple strategy improvements. Learn the Artificial Hummingbird Algorithm. Its core improvement strategies include:

[0113] 1) Population initialization strategy based on optimal point set

[0114] In the original AHA algorithm, population initialization relies entirely on random generation, which easily leads to uneven distribution of candidate solutions in the search space. This uneven distribution manifests primarily as an over-density of solution individuals in some regions, while other regions are relatively sparse. This not only weakens the global exploration ability during the guided foraging and domain foraging phases but may also disrupt the balance between global exploration and local development due to insufficient population diversity. This defect is particularly prominent in complex optimization problems such as multi-robot path planning. In the solution space of path planning, if the initial population distribution quality is poor, it can easily lead to slow algorithm convergence, low-quality paths, or even premature entrapment in local optima, resulting in path search stagnation and severely affecting the efficiency and safety of the final path planning. The optimal point set method based on the uniform distribution theory in number theory can achieve efficient and uniform sampling of the solution space through specific mathematical constructions (such as...). Figure 1 (As shown). Applying the optimal point set to the population initialization stage of AHA can significantly improve the diversity and distribution uniformity of the initial population, thereby effectively improving the balance between exploration and exploitation in the subsequent optimization process, and thus effectively alleviating the problems caused by poor initialization.

[0115] Let the population size be The decision space dimension is Then the formula for generating the optimal point set is:

[0116] (7)

[0117] In the formula, For the best point, take , , To meet The smallest prime number. According to formula (7), the population generation formula is:

[0118] (8)

[0119] In the formula, For the first Individual in the first The position of the dimension For the best point of concentration Individual in the first Dimension value, and The first The lower and upper limits of values ​​for each dimension.

[0120] 2) Elite-led mutation migration foraging strategy

[0121] During the migration and foraging phase of AHA, as can be seen from equation (3), the location of a new individual is determined by only a random food source. It fails to effectively utilize information such as the best individuals in the current population or the global search history, resulting in a lack of guidance in the search process.

[0122] Introducing mutation behavior guided by elite individuals, the mutation vector is first generated:

[0123] (9)

[0124] In the formula, For migrating individuals, Scaling factor The globally optimal elite individual For each random individual in the population, binomial mutation is then performed:

[0125] (10)

[0126] In the formula, The variability rate, The first of the variant vectors The dimension value is rand, which is a random number between 0 and 1. By introducing elite individuals, migrating individuals are made to move closer to elite individuals, while random individuals and mutation behaviors are introduced to generate random perturbations.

[0127] 3) Learning to select adaptive foraging strategies

[0128] Learning is a model-free reinforcement learning algorithm whose core principle is to learn... The value function of a function guides the agent to make optimal decisions in the environment. Learning includes Value table, status ,award and actions Four elements. The agent identifies the state from the environment. Then based on Choose a promising action from the value table. Execute. After this action is executed, the environment will transition to the current state. and return the reward to the agent. , The value table will be updated using formula (11):

[0129] (11)

[0130] In the formula, In the state Next action Corresponding value; The learning rate is used to adjust the relative weights of historical experience and immediate rewards to the current estimate. The discount rate measures the degree to which future earnings diminish relative to immediate earnings. For state Next action The rewards received.

[0131] Traditional AHA algorithms employ a fixed probabilistic mechanism to select foraging strategies, failing to consider the dynamic impact of different strategies on population evolution trends and future search directions. This static selection approach, lacking real-time perception and evaluation of the search environment and population state, is prone to premature convergence, causing the population to fall into local optima and hindering the generation of diverse new individuals. Consequently, it weakens its robustness and solution efficiency in complex scenarios such as multi-robot path planning.

[0132] To overcome the above-mentioned defects, the present invention proposes to... The learning mechanism is deeply integrated with the AHA algorithm, endowing it with self-learning and dynamic decision-making capabilities. By introducing reinforcement learning, the algorithm can comprehensively consider historical experience, current state, and future rewards in each decision, achieving adaptive selection of foraging strategies and thus reaching a better dynamic balance between global exploration and local exploitation. Figure 2 As shown, the proposed The learning framework consists of three core modules: the environment module (corresponding to the solution space and constraints of the multi-robot path planning problem), and the environment module. The learning module (responsible for strategy evaluation and selection) and the reinforcement interaction process (implementing status feedback and...) (Value update). The following sections will explain in turn. Value table design method, based on The learning strategy adaptive selection mechanism and the complete execution flow of the algorithm.

[0133] Learning state settings: The overall search state of the population can be effectively represented by the update status of individual fitness values. Therefore, the update status of individual fitness values ​​in the population is used as... The learned state signals are used to guide individuals in choosing between guided foraging and territorial foraging strategies. Specifically, the population state is defined. The difference between the number of individuals whose fitness values ​​were updated and the number of individuals whose fitness values ​​were not updated:

[0134] (12)

[0135] In the formula, This represents the number of individuals whose fitness value is improved (i.e., their position is updated) in the current iteration. This represents the number of individuals whose fitness values ​​have not been increased (positions not updated). Because... The value table requires the states to be in discrete form, so continuous states need to be represented as discrete states. Normalization Normalization to discrete state space:

[0136] (13)

[0137] In the formula, 0 indicates that the population as a whole tends to stagnate, and 1 indicates that the population is still actively improving.

[0138] Learning Action Settings: To achieve adaptive selection of foraging strategies, two actions are defined to dynamically adjust the probability of strategy selection. They are respectively (Increase the probability of guiding foraging) and (Increases the probability of foraging in the territory). Each action corresponds to a probability adjustment amount. . set up and They represent the first In the next iteration, the individual chooses the probability of being guided to food and foraging in the territory, and satisfies the following conditions: In each iteration, the system selects an action based on the current state and updates accordingly. and To ensure that both foraging strategies have a chance of being used in the long-term search, probability boundary constraints are set. and , respectively represented as and The lower and upper bounds of allowed values. Taking the probability of guiding foraging as an example, its update process can be expressed as:

[0139] (14)

[0140] Learning reward system: A reasonable reward mechanism can effectively guide hummingbirds' foraging behavior and accelerate the optimization process. Therefore, this algorithm designs reward signals based on changes in population state. Its definition is:

[0141] (15)

[0142] In the formula, This represents the normalized discrete population state (taking values ​​of 0 or 1). The reward design aims to intuitively reflect the population's search performance. If... This indicates that more individuals in the population have updated their fitness than those that haven't, meaning the population is in a state of effective improvement, and the algorithm provides a positive reward to encourage the current strategy; if This indicates that the population update has stagnated or degenerated, meaning it may be trapped in a local optimum. The algorithm provides zero reward or negative incentive to prompt policy adjustment.

[0143] Value table update: Based on the above Regarding the design of states, actions, and rewards in the learning process, the algorithm executes the following in each iteration cycle: Learning and updating process:

[0144] ① State judgment: Determine the discrete state of the current population based on equations (12) and (13). ;

[0145] ② Action selection: based on Greedy strategy, from Select state from value table The next corresponding maximum The action of value;

[0146] ③ Strategy Execution and State Update: Execute the selected action, dynamically adjust the probability of guided foraging and territorial foraging, and then perform a complete AHA iteration to obtain a new generation of the population and calculate the new state. ;

[0147] ④ Reward Calculation: Calculate the immediate reward obtained from this state transition according to formula (15). ;

[0148] ⑤ Value update: based on Learn the update rules (as shown in equation (11)) and combine them with the current state. Selected action ,award And new status ,renew The corresponding entry in the value table.

[0149] Through the aforementioned closed-loop interaction, the algorithm can gradually learn which foraging strategy should be prioritized under different population states, thereby achieving adaptive dynamic control of the search process. In this algorithm... The value table is generated using a random initialization method.

[0150] Table 1 Value table

[0151]

[0152] Table 1 lists the algorithms used in this paper. Value table structure, in the table Indicates the state Take action below The corresponding long-term cumulative reward valuation. It should be noted that during the optimization process, the entire population shares the same... Value table, and action selection follows Greedy strategies aim to balance exploitation and exploration.

[0153] This invention employs a best-point set strategy to initialize the population, effectively improving the diversity and spatial uniformity of initial solutions. Furthermore, it introduces an elite-guided mutation migration foraging strategy, utilizing the current best individual (elite) to guide the search direction, significantly enhancing the algorithm's convergence speed and local exploitation capability. Simultaneously, by introducing random individuals and adaptive mutation operations to generate controllable perturbations, it maintains population diversity while ensuring global exploration capability. In addition, each hummingbird is modeled as... The learning agent dynamically updates the selection probability of each foraging strategy based on the population's positional evolution, achieving adaptive strategy adjustment. This fundamentally overcomes the shortcomings of the original AHA algorithm, such as fixed strategy selection, lack of adaptability, and susceptibility to local optima. The overall algorithm flow is as follows: Figure 3 As shown.

[0154] Time complexity analysis of MQAHA: The time complexity of MQAHA mainly depends on initialization, population size (n), and maximum number of iterations (n). ) and problem dimensions ( The initialization phase employs a best-point-set strategy, with a time complexity of O(n log n). In each iteration, the computational complexity of position updates (including guided foraging and territorial foraging) is O(n). The complexity of the elite-guided mutation migration feeding strategy is Therefore, the overall time complexity of MQAHA is: Therefore, the overall time complexity of MQAHA remains consistent with the original AHA algorithm, and the proposed improvement strategy does not introduce additional asymptotic time overhead.

[0155] Numerical optimization experiment analysis:

[0156] a) Experimental environment

[0157] To verify the optimization performance of MQAHA, the CEC2017 test set was used, along with DBO from the literature (Xue J, Shen B. Dung beetle optimization: a novel metaheuristic algorithm for global optimization[J]. Supercomputing Journal, 2023, 79(7): 7305-7336), GWO from the literature (Mirjalili S, Mirjalili SM, Lewis A. Gray wolf optimizer[J]. Advances in Engineering Software, 2014, 69: 46-61), WOA from the literature (Mirjalili S, Lewis A. T whale optimization algorithm[J]. Advances in Engineering Software, 2016, 95: 51-67), and WOA from the literature (Lee SW, Haider A, Rahmani AM, et al. A review of beetle optimization algorithms and their variants: statistical analysis, progress and structural review[J]. Review of Computer Science, 2025, 57: A comparative experiment was conducted using BWO from 100740), HHO from the literature (Heidari AA, Mirjalili S, Faris H, et al. Harris Eagle Optimization Algorithm: Algorithm Principles and Applications [J]. Future Generation Computer Systems, 2019, 97: 849-872), PSO from the literature (Gad A G. Particle Swarm Optimization Algorithm and Its Applications: A Systematic Review [J]. Archives of Engineering Computation Methods, 2022, 29(5): 2531-2561), and the original AHA algorithm. The parameter settings for each algorithm are shown in (2). To ensure the effectiveness of the experiment, the population size of all comparative algorithms was uniformly set to 30, and the maximum number of iterations was set to 1000. To minimize the impact of algorithm randomness on the results, each algorithm was run independently 30 times. The experiment was implemented on the MATLAB R2023b platform. The test environment was a computer with an AMD Ryzen 5 3600 six-core processor and 8GB of memory, and the Windows 11 operating system.

[0158] Table 2 Algorithm Parameter Settings

[0159]

[0160] b) CEC2017 Test Function Experimental Analysis

[0161] The experiment used the CEC2017 test function set to systematically evaluate the algorithm performance. This test set contains 29 functions across four main categories, covering the following typical problem types: unimodal functions (F1, F3), used to measure the convergence speed of the algorithm approaching the unique global optimum under limited computational resources; multimodal functions (F4-F10), designed to test the algorithm's ability to escape local optima and locate the global optimum; and mixed functions (F11-F20) and combined functions (F21-F30), which together constitute complex, nonlinear, high-dimensional search scenarios to comprehensively evaluate the algorithm's dynamic balance performance between global exploration and local exploitation. Table 3 shows the optimization results (mean ± standard deviation) of each comparative algorithm on representative test functions. Figure 4 The curves in the figure show a comparison of convergence performance on typical functions.

[0162] Table 3 CEC2017 Test Results

[0163] The experimental results in Table 3 lead to the following analysis: MQAHA achieved optimal convergence accuracy on unimodal functions F1 and F3. Although PSO, AHA, and QMAHA achieved the same convergence accuracy as MQAHA on F3, MQAHA exhibited higher stability (smaller standard deviation). This indicates that MQAHA possesses strong local exploitation capabilities in search spaces with explicit gradient information. On multimodal functions F4-F10, MQAHA achieved optimal convergence accuracy on F5, F6, F7, and F9. This demonstrates that MQAHA can effectively overcome premature convergence problems when facing multiple local extrema, exhibiting good global exploration capabilities. On mixed functions F11-F20, MQAHA achieved optimal convergence accuracy on seven test functions: F11-F15, F19, and F20. Its outstanding performance on complex structure functions verifies the algorithm's strong adaptability and robustness in nonlinear, high-dimensional mixed search spaces, proving the effectiveness of the proposed strategy. On combinatorial functions F21-F30, MQAHA also achieves the best convergence accuracy on F23, F24, F26, F27, and F29. This further demonstrates that the algorithm still possesses excellent comprehensive solution capabilities in highly complex, multi-mode combinatorial optimization problems. Furthermore, Figure 4 The convergence curves intuitively show that MQAHA not only converges faster on typical test functions (such as F12, F13, etc.), but also has a significantly better final convergence accuracy than other comparative algorithms, further confirming its excellent ability to balance global exploration and local development.

[0164] In summary, MQAHA performs excellently on unimodal, multimodal, mixed, and combined functions, achieving fast, stable, and high-precision convergence. This demonstrates that the algorithm effectively balances exploration and development, and possesses the capability to solve complex spatial search problems such as path planning for multiple mobile robots.

[0165] The implementation steps of MQAHA are as follows:

[0166] Step 5.1: Generate initial candidate solutions: ;

[0167] Step 5.2: Evaluate the objective function value using formula (21);

[0168] Step 5.3, according to The value table adjusts the probabilities of guided foraging and territorial foraging, and then selects guided foraging or territorial foraging based on the foraging probability;

[0169] Step 5.4: Guide the selected foraging behavior to engage in corresponding foraging or territorial foraging.

[0170] Step 5.5: Update the population and population based on the results of Step 5.4. Value table;

[0171] Step 5.6: Determine whether the MQAHA termination condition is met. If it is met, update the current position list and return to step 4; otherwise, return to step 5.3.

[0172] The elite-guided mutation migration foraging method in step 5.4 is as follows:

[0173] Introducing mutation behavior guided by elite individuals, the mutation vector is first generated:

[0174] (9)

[0175] In the formula, For migrating individuals, Scaling factor The globally optimal elite individual For each random individual in the population, binomial mutation is then performed:

[0176] (10)

[0177] In the formula, The variability rate, The first of the variant vectors The dimension value is rand, which is a random number between 0 and 1. By introducing elite individuals, migrating individuals are made to move closer to elite individuals, while random individuals and mutation behaviors are introduced to generate random perturbations.

[0178] In step 5.5 The value table will be updated using formula (11):

[0179] (11)

[0180] In the formula, In the state Next action Corresponding value; The learning rate is used to adjust the relative weights of historical experience and immediate rewards to the current estimate. The discount rate measures the degree to which future earnings diminish relative to immediate earnings. For state Next action The rewards received.

[0181] Performance Analysis of Multi-Mobile-Robot Path Planning: To evaluate the effectiveness of MQAHA in solving the multi-mobile-robot path planning problem, this invention conducted a series of simulation comparison experiments. DBO, WOA, BWO, HHO, and AHA were selected as comparison algorithms. The complete planning process of MQAHA for this problem is as follows: Figure 5 As shown.

[0182] The experiment was set up with two 100cm sections. A 2D path planning scenario with a height of 100cm. Figure 6 Scenario 1 contains three types of obstacles: blue circles and gray rectangles are static obstacles, and dashed lines represent the movement trajectories of dynamic obstacles. Scenario 2, while keeping the obstacle types the same, increases the number of static rectangular obstacles, as well as the number of dynamic obstacles and robots, thus significantly increasing the complexity of the search space and placing higher demands on the algorithm's ability to balance exploration and development.

[0183] The ranges for robot speed and steering angular velocity are respectively set as follows: cm / s and rad / s. The algorithm performance was evaluated by combining the total path length and the number of algorithm steps. All algorithms were evaluated a maximum of 1000 times in the simulation, the population size was set to 30, and the other parameters are set as shown in Table 2.

[0184] In scenario 1, the number of robots is set to 6, and the number of dynamic obstacles is 3. Figure 7 The path planning results of each algorithm are shown. Tables 4 and 5 respectively summarize the path length and number of steps for each algorithm. Figure 7 It is evident that the paths planned by WOA, AHA, and MQAHA are relatively smooth. Further analysis of Tables 4 and 5 shows that MQAHA outperforms AHA and other comparative algorithms in path length for robots 1, 3, 4, 5, and 6. In terms of total path length, MQAHA achieves the shortest path, shortening it by 0.89% compared to AHA; and in terms of the number of steps, MQAHA reaches the target point with the fewest steps in the planning of all six robots.

[0185] The results show that MQAHA has stronger path optimization capabilities, higher convergence efficiency, and better multi-robot cooperative obstacle avoidance performance in complex dynamic environments, demonstrating significant advantages in multi-mobile robot path planning tasks.

[0186] Table 4: Statistics of Path Length in Scenario 1

[0187]

[0188] Table 5: Statistical Table of Path Planning Steps in Scenario 1

[0189]

[0190] In scenario 2, the number of robots increases to 8, and the number of dynamic obstacles increases to 5. Figure 8 The path planning results of each algorithm in this complex environment are visually demonstrated. Due to the significantly increased environmental complexity, DBO, WOA, BWO, HHO, and AHA all exhibit varying degrees of local convergence when avoiding dense rectangular obstacles. Specifically, the path points tend to converge in specific regions (such as...). Figure 8 The areas marked with red circles in the image show excessive concentration, reflecting the algorithm's insufficient exploration ability under complex constraints. In contrast, the path planned by MQAHA not only avoids point accumulation but also exhibits a smoother and more continuous overall trajectory.

[0191] Further analysis of the data in Tables 6 and 7 shows that MQAHA achieves the shortest paths on robots 1, 2, 3, 4, 7, and 8, with a total path length reduced by 3.45% compared to AHA. Simultaneously, it achieves the fewest running steps on robots 1, 2, 4, 7, and 8, and its total number of steps is also the lowest among all algorithms. These results fully demonstrate that MQAHA can effectively handle highly complex search environments and successfully achieve efficient collaborative path planning for multiple mobile robots, thus verifying the effectiveness and practical application value of the improved strategy proposed in this invention.

[0192] Table 6 Statistics of Path Length in Scenario 2

[0193]

[0194] In summary, this invention proposes an improved Artificial Hummingbird Algorithm (MQAHA) that integrates multiple strategies to address the path planning problem for multiple mobile robots in complex environments. The core improvements of this algorithm include: First, initializing the population using a best-point set method to enhance the diversity and uniformity of initial solutions; Second, designing an elite-guided mutation migration foraging strategy that integrates the guidance of elite individuals with the perturbation of random individuals, thereby improving local development efficiency and global exploration potential; Third, introducing... The learning mechanism adaptively adjusts the probability of foraging strategy selection based on the population's search status, achieving a dynamic balance between exploration and exploitation, and effectively suppressing premature convergence.

[0195] For performance evaluation, MQAHA was first systematically tested on the CEC2017 test function set. Experiments showed that MQAHA achieved the best convergence accuracy on 18 out of 29 test functions, and significantly outperformed the comparison algorithms in convergence speed. Furthermore, simulation experiments were conducted using MQAHA in two multi-mobile robot path planning scenarios with different complexities. Results showed that MQAHA can generate smooth and safe cooperative paths in both environments, exhibiting excellent performance in both total path length and number of steps. Compared to the original AHA, MQAHA shortened the path by 0.89% and 3.45% in scenarios 1 and 2, respectively, verifying the effectiveness and practicality of the proposed strategy in improving path quality and optimizing search efficiency.

[0196] Table 7 Statistics of Path Planning Step Length in Scenario 2

[0197]

Claims

1. A multi-robot path planning method in a complex environment based on MOAHA, characterized in that, The method comprises the following steps: Step 1, constructing the robot motion model, assuming that all robots are uniform circular rigid bodies, whose positions at time t are calculated by the following formula (15): t = t + Δt （15）， wherein is the next position coordinate of the robot, is its current position coordinate; is the velocity and is the radial angle of the robot, respectively; is the time the robot moves at the velocity v. Step 2, constructing a comprehensive evaluation function based on the above motion model, the comprehensive evaluation function is used for evaluating the comprehensive optimization of path length, obstacle avoidance ability and anti-collision performance among robots in the cooperative motion process of the multiple robots, the comprehensive evaluation function adopts a weighted summation manner to construct a total fitness function, and a calculation formula of the comprehensive evaluation function is as shown in the following: （21）， In the formula, fitness function for the shortest distance of the robot to reach the target point, fitness function for avoiding collision with obstacles, fitness function for avoiding collision with other robots; Step 3, initializing a current position list of all mobile robots and a corresponding target position list; Step 4, judging whether all robots have arrived at respective target points, if all arrive, the path planning is ended, otherwise, entering step 5; Step 5, the next motion state of each robot is determined by MQAHA, and its current position is updated, and then it returns to step 4, wherein the MQAHA is based on the speed and radial angle of all robots, i.e. the input variables are: wherein and the search space ranges are and the robot will move to the feasible position in turn until it reaches the target point, and the MQAHA will learn and combine with the AHA algorithm to give the AHA self-learning ability; through comprehensive consideration of the historical state, current state and future state of the population, adaptive selection of foraging strategies is realized.

2. The method for multi-robot path planning in complex environment based on MQAHA according to claim 1, wherein, Fitness function for robot to reach target point in shortest distance is represented as: （16）， In the formula, For the first A robot Position coordinates at that moment For the first The target position coordinates of the robot.

3. The method for multi-robot path planning in complex environment based on MQAHA according to claim 1, wherein, Fitness function to avoid collision obstacles is represented as: （17）， In the formula, is a penalty value, is a safe collision distance, is the sum of distances between the first robot and the obstacles, and the specific calculation formula is as follows: （18）， wherein and is the number of robots and obstacles, respectively, is the position of the th obstacle, is the position of the th robot.

4. The method for multi-robot path planning in complex environment based on MQAHA according to claim 1, wherein, Fitness function to avoid collisions with other robots is represented as: （19）， In the formula, and have the same meaning as in the function , is the sum of the distances between the robots, and the specific calculation formula is as follows: （20）， wherein is the position of the th robot, and , is the position of the th robot.

5. The method for multi-robot path planning in complex environment based on MQAHA according to claim 1, wherein, The implementation steps of the MQAHA are as follows: Step 5.1, generating an initial candidate solution: ; Step 5.2, evaluating the objective function value through formula (21); Step 5.3, according to The value table adjusts the guided foraging and the field foraging probability, and then selects the guided foraging or the field foraging according to the foraging probability; Step 5.4, performing corresponding guiding foraging or field foraging according to the selected foraging behavior; Step 5.

5. Update the population and value table; Step 5.6, judging whether the MQAHA termination condition is met, if yes, updating the current position list and returning to step 4, otherwise, returning to step 5.

3.

6. The method for multi-robot path planning in complex environment based on MQAHA according to claim 5, wherein, The MQAHA adopts a good point set method when initializing a population; the specific operation steps of the good point set initialization are as follows: Let the population size be , the decision space dimension be , then the optimal point set generation formula is: （7）， In the formula, For the best point, take , , To meet The minimum prime number according to formula (7), the generation formula of the population is: （8）， In the formula, is the first dimensional position of the individual, is the first dimensional value of the individual in the optimal point, is the first dimensional value of the individual in the optimal point, and are the lower limit and the upper limit of the first dimensional value, respectively.

7. The method for multi-robot path planning in complex environment based on MQAHA according to claim 5, wherein, The mutation migration foraging method of the elite guiding in step 5.4 is as follows: The mutation vector is first generated based on the mutation behavior guided by the elite individual: （9）， wherein is the migration individual, is the scaling factor, is the global optimal elite individual, is a random individual in the population, on which a binomial mutation operation is performed: （10）， In the formula, is the mutation rate, is the first dimension value of the mutation vector, is a random number between 0 and 1, by introducing the elite individual, let the migration individual close to the elite individual, and introduce the random individual and the mutation behavior to produce random disturbance.

8. The method for multi-robot path planning in complex environment based on MQAHA according to claim 4, wherein, In step 5.5 The value table will be updated by equation (11): （11）， In the formula, is the value corresponding to the action performed in the state ; ; is the learning rate, used to regulate the relative weight of historical experience and immediate reward on the current estimate; is the discount rate, used to measure the degree of decay of future income relative to immediate income, is the reward obtained by performing the action in the state .

Images