Hybrid optimization algorithm for collaborative scheduling and dynamic obstacle avoidance of robot cluster

Abstract

The invention relates to the technical field of artificial intelligence and intelligent robot cluster control, in particular to a hybrid optimization algorithm for robot cluster collaborative scheduling and dynamic obstacle avoidance. Specifically, the invention designs a novel hybrid optimization framework named as GYPSO-PPO based on the fusion of an improved adaptive hierarchical particle swarm optimization algorithm and a near-end policy optimization algorithm (Proximal Policy Optimization, PPO). The framework aims to solve the key technical problems of low convergence speed, easiness in falling into a local optimal solution, unstable strategy update, hyperparameter sensitivity and the like of deep reinforcement learning in a complex environment and a high-dimensional state space. The method is especially suitable for solving the problem of environment dynamic complexity caused by state fluctuation and environment change of a multi-agent cluster in a complex environment, and can be widely applied to real-time intelligent control systems needing continuous decision making, such as path planning, dynamic obstacle avoidance and multi-machine cooperative scheduling of cluster unmanned aerial vehicles, warehouse logistics robots and the like.

Description

Technical Field

[0001] This invention belongs to the field of intelligent robot swarm control technology, specifically involving a hybrid optimization algorithm (GYPSO-PPO) based on adaptive hierarchical particle swarm optimization and near-end strategy optimization for robot swarm collaborative scheduling and dynamic obstacle avoidance. It can be widely applied to real-time intelligent control systems that require continuous decision-making, such as path planning, dynamic obstacle avoidance, and multi-machine collaborative task scheduling for UAV swarms, warehouse logistics robots, etc. Background Technology

[0002] Deep Reinforcement Learning (DRL), as an important branch of machine learning, learns optimal decision-making policies through the interaction between agents and the environment, and has made groundbreaking progress in many fields. The Proximal Policy Optimization (PPO) algorithm has become one of the most mainstream policy gradient algorithms due to its excellent stability and sample efficiency. PPO limits the magnitude of policy updates by introducing pruning to replace the objective function and importance sampling, effectively avoiding the collapse phenomenon during training. However, PPO still relies on gradient-based optimization methods, which limits its search capability to initial parameters and local gradient information. When dealing with multimodal, non-convex high-dimensional optimization problems, there is still a risk of convergence to a suboptimal solution.

[0003] Particle Swarm Optimization (PSO) is a swarm intelligence global optimization algorithm derived from simulations of avian social behavior. It finds optimal solutions through information sharing and cooperation among individuals in the swarm, exhibiting strong global exploration capabilities. Traditional PSO algorithms are simple in structure, have few parameters, and are easy to implement, but they also suffer from drawbacks such as premature convergence and insufficient search accuracy in later stages. In recent years, although some studies have attempted to combine PSO with Dynamic Range Optimization (DRL) to utilize PSO's global search capabilities to optimize neural network parameters or strategies, these methods often employ simplistic combinations that fail to fully consider the structural and diverse nature of the PSO population, as well as the deep synergy between PSO and DRL training processes, resulting in limited performance improvements.

[0004] In modern multi-agent systems, robots must perform tasks in complex and dynamic environments, and their path planning and dynamic obstacle avoidance capabilities directly impact system efficiency and safety. Traditional path planning algorithms (such as A, D, and RRT) often perform poorly in highly dynamic, multi-obstacle environments, struggling to achieve real-time, smooth, and safe decision-making. In recent years, deep reinforcement learning (DRL) has been introduced into this field, but it still suffers from slow convergence and a tendency to get trapped in local optima. Therefore, there is an urgent need for an intelligent decision-making framework that combines global optimization and local fine-tuning to improve the robot's autonomous decision-making capabilities in complex environments. Summary of the Invention

[0005] To achieve the above objectives, this invention proposes a hybrid optimization algorithm for collaborative scheduling and dynamic obstacle avoidance in robot swarms, comprising the following steps:

[0006] S1. Initialize the main policy network of the near-end policy optimization (PPO) algorithm. and value network and initialize a containing A particle swarm of 100 particles, where the position vector of each particle represents a set of parameter values ​​for a PPO policy network.

[0007] S2, in the environment The current main policy network is running in the middle. Collect state, action, and reward sequence data, and calculate the advantage function estimate;

[0008] S3. Using the collected data, perform K rounds of optimization on the PPO pruning replacement objective function using stochastic gradient descent, and update the main policy network parameters. and value network parameters ;

[0009] S4. Determine if the GYPSO intervention conditions are met: If met, proceed to step S5; otherwise, return to step S2 to continue the next round of PPO training.

[0010] S5. Set the parameters of the current PPO master policy network. Assign a value to a specified particle in the particle swarm, and calculate the fitness value of each particle based on the performance of the strategy it currently represents in the environment.

[0011] S6. Execute the Adaptive Hierarchical Particle Swarm Optimization (GYPSO) algorithm: Particles are stratified according to their fitness values, and different velocity and position update strategies are adopted for particles at different levels. Simultaneously, the algorithm parameters are adaptively adjusted based on the population evolutionary state, and a hybrid mutation operation is performed. This process is iteratively optimized for T generations to find the global optimum. ;

[0012] S7. Find the globally optimal solution from GYPSO. The parameter vector represented is assigned to the main policy network of PPO. This completes the global optimization and update of the strategy parameters;

[0013] S8. Return to step S2 and continue the PPO data collection and training process until the preset total number of training rounds or performance threshold is reached.

[0014] Preferably, the adaptive parameter adjustment strategy in step S6 specifically includes: inertia weights. Individual learning factors Social learning factors All vary with the number of iterations Dynamic changes; inertial weights This is used to balance the global exploration and local exploitation capabilities of the algorithm, and its value ranges from the maximum value. Decreasing linearly to the minimum value The calculation formula is: ,in Maximum number of iterations; individual learning factor This is used to control how much a particle follows its historical best position, and its value ranges from a maximum value. Decreasing linearly to the minimum value The calculation formula is: Social learning factor This is used to control how much a particle follows the group's historical best position; its value ranges from a minimum. linearly increasing to the maximum value The calculation formula is: This parameter setting allows the algorithm to focus on global exploration in the early stages and on detailed local development in the later stages.

[0015] Preferably, the hierarchical particle update strategy in step S6 specifically includes: at the beginning of each generation of optimization, all particles are sorted in ascending order according to their fitness values, the top 20% of particles are classified into a high-quality particle group (GP), the middle 60% into a normal particle group (CP), and the bottom 20% into a low-quality particle group (BP); for particles in the low-quality particle group (BP), their updates no longer refer to their own historical best values. Instead, it learns randomly from a particle in the high-quality particle swarm GP, and its velocity update formula is: ,in Indicates the first The position of a particle is randomly selected from a high-quality particle swarm, and ω represents the inertia coefficient. This represents the current position of the i-th particle in the search space at the t-th iteration.

[0016] Preferably, in step S6, a competitive learning strategy is adopted for particles in the high-quality particle swarm GP: for the first particle in the GP... One particle, randomly selected from GP, is different from... particles Compare the fitness of the two; if the particles If its fitness is better, then it follows the formula An update is performed, which involves moving in the direction of the difference between its own position and the random particle's position to enhance its local exploitation capabilities in high-quality regions; among which, This represents the position of the i-th particle in the high-quality particle swarm at the t-th iteration. This represents the position of a randomly selected particle other than particle i within the same high-quality particle swarm.

[0017] If particles If the fitness is better, then the particle Return to the standard PSO update formula To maintain population diversity and avoid overcrowding; among them, , They are two random numbers between 0 and 1. The best historical position of the i-th particle in the t-th iteration. This represents the global optimal value at the t-th iteration.

[0018] Preferably, in step S6, the update strategy for particles in the ordinary particle swarm CP integrates information from the global optimum and elite particles; an elite particle swarm EP is introduced, which consists of particles that exhibit continuous optimization potential during evolution; when ordinary particles update their velocity, their social learning component no longer only aims at the global optimum. Instead, it moves towards a hybrid sample generated by random crossover operations between the global optimal solution and elite particles. The speed update formula is: ;in, This represents the component of the historical best position of the i-th particle in the d-th dimension at the t-th iteration. This represents the current position of the i-th particle in the d-th dimension at the t-th iteration. It represents the position of the cross-particle in the d-dimensional dimension at the t-th iteration. These cross-particles are generated through random cross-operations.

[0019] Mixed samples Each dimension The corresponding dimension is randomly selected from the elite particle EP or the high-quality particle GP (i.e., the population containing the global optimal solution) based on probability. , A random number between [0,1]; It is the position of the elite particle swarm in the d-th dimension. It is the position of the high-quality particle swarm in the d-th dimension.

[0020] Preferably, the hybrid mutation mechanism in step S6 specifically involves: continuously monitoring the global optimal solution during the GYPSO iteration process. Update status; if If no improvement is achieved after K consecutive generations, the population is considered to be trapped in a local optimum. In this case, a Cauchy mutation operation is applied to the particles in the current population with a certain probability. The mutation formula is: ,in Random numbers generated from a standard Cauchy distribution. This is the scaling factor; It refers to the new position a particle occupies in the next generation after undergoing a random mutation. It is the position before the mutation.

[0021] Preferably, the GYPSO intervention conditions in step S4 are as follows:

[0022] (1) Performance plateau judgment: during continuous Within a PPO training cycle, the average reward gain of the policy in the test environment is less than a preset threshold. ;

[0023] (2) Dynamic Interval Intervention: The current cumulative number of PPO training cycles reaches the dynamically calculated intervention interval. If any condition is met, proceed to step S5; otherwise, return to step S2 to continue the next round of PPO training.

[0024] Wherein, the intervention interval From the basic interval Environmental complexity index The calculation is dynamically derived, and the formula is as follows: In the formula This is the proportionality coefficient. This is a preset minimum interval threshold used to prevent excessive intervention.

[0025] Preferably, the objective function optimized by GYPSO in step S6 is the objective that the PPO algorithm aims to maximize, and its core is to tailor the alternative objective function. For a parameter particle Its fitness value By running the policy network it represents in the environment And evaluate it by calculating its expected cumulative reward, or by calculating it in the current PPO's experience data buffer with parameters. The policy network constructed corresponds to The value is used for approximate evaluation; this design ensures that the optimization direction of GYPSO is completely consistent with the ultimate goal of PPO.

[0026] Preferably, the environmental complexity index The calculation is based on one or more of the following factors: the number of dynamic obstacles perceived per unit time, the average congestion level in path planning, and the length of the task queue.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] 1. It fundamentally avoids local optima: By actively exploring the policy parameter space through the periodic global search of the GYPSO algorithm, it effectively overcomes the inherent defects of the pure gradient descent method and significantly improves the performance ceiling of the final policy.

[0029] 2. Significantly improves convergence efficiency: GYPSO can quickly locate high-performance regions, providing PPO with a superior starting point and direction for optimization, reducing the interactive data and training time required to achieve the same performance, and making training faster.

[0030] 3. Enhanced algorithm robustness: The adaptive parameter strategy reduces the algorithm's sensitivity to hyperparameter settings. Hierarchical and mutation mechanisms enhance its adaptability to environmental randomness, exhibiting more stable performance across different tasks.

[0031] 4. Achieving efficient exploration-exploitation balance: This framework achieves temporal and spatial separation and coordination between exploration (global search of GYPSO) and exploitation (local fine-tuning of PPO), resulting in higher search efficiency and more intelligent decision-making.

[0032] 5. Achieved adaptive matching of computing resources and environmental requirements: Through an innovative dynamic intervention mechanism, the triggering frequency of GYPSO global optimization is linked with the environmental complexity in real time. When the environment is complex, the global search capability is enhanced, and when the environment is simple, the computational overhead is saved. Thus, while ensuring the performance of the algorithm, the computational efficiency and practicality of the entire system are significantly improved. Attached Figure Description

[0033] Figure 1 This is a structural diagram of the GYPSO-PPO algorithm of the present invention, used to illustrate the overall structure of the GYPSO-PPO framework, including the integration mechanism of particle swarm optimization (PSO) and proximal policy optimization (PPO).

[0034] Figure 2 This is a flowchart of the GYPSO-PPO algorithm of the present invention, used to illustrate the algorithm flow of GYPSO-PPO. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but the embodiments of the present invention are not limited thereto. Those skilled in the art should understand that the following embodiments are only used to clearly illustrate the technical principles and implementation process of the present invention, and are not intended to limit the scope of protection of the present invention. Any simple modifications, equivalent changes, and alterations made based on the essential spirit of the present invention should be included within the scope of protection of the present invention.

[0036] Example 1: Intelligent UAV Swarm Task Allocation System Based on GYPSO-PPO

[0037] This embodiment takes the task allocation, path planning and dynamic obstacle avoidance of UAV swarm as the application scenario, builds a simulation environment on the Python and PyTorch platforms, and uses Gazebo or ROS for physical simulation verification.

[0038] S101: System Initialization

[0039] First, initialize the algorithm hyperparameters and the network:

[0040] Setting the main hyperparameters of the PPO algorithm: learning rate Cutting factor Value function coefficients Entropy reward coefficient The value of the time steps for each batch of data collection and the maximum number of iterations. The same applies here, 2048, representing the number of training rounds. The state space is defined as the robot's current position, velocity, and sensor data (LiDAR, vision); the motion space is defined as linear velocity and angular velocity; the reward function is designed as follows: a reward of +100 for successfully reaching the target point, a penalty of -100 for collisions, and a penalty of -0.1 for energy consumption per step.

[0041] Initialize policy network and value network The policy network is a two-layer fully connected neural network (the input layer has the same dimension as the state space, the hidden layer has 64 neurons, and the output layer is the action distribution parameters). The value network has a similar structure but its output is a single scalar.

[0042] Set the GYPSO algorithm parameters (as shown in Table 1): number of particles The optimization interval is set to start once every 5 PPO training cycles, with a maximum number of iterations. Inertial weight range Learning factor range Cauchy variation scaling factor Stagnation-triggered algebra .

[0043] Initialize particle swarm: randomly generated There are 1 particle, and the position vector of each particle. Dimensions and Policy Networks parameter vector The dimensions are the same. The current PPO policy network parameters... Assign the value to one of the particles.

[0044] Table 1: GYPSO Algorithm Parameter Settings

[0045]

[0046] S102: Data Collection and PPO Update

[0047] Run the current policy in the CartPole environment. The value of the collection length and the maximum number of iterations Same trajectory data This includes status, actions, rewards, and termination indicators.

[0048] The advantage function value at each step is calculated using the generalized advantage estimation (GAE) method. .

[0049] Using the collected data, randomly select mini-batches of data and apply gradient descent algorithms (such as the Adam optimizer) to the PPO's objective function. And the error term of the value function Round optimization, updating policy network parameters and value network parameters .

[0050] S103: GYPSO intervention judgment

[0051] After each PPO training cycle, perform the following operations:

[0052] (1) Environmental complexity assessment: Based on sensor data within the most recent time window (e.g., 100 training steps), calculate the environmental complexity index. In this embodiment, It is simplified to the normalized value of the number of times dynamic obstacles are detected by the lidar within this time period.

[0053] (2) Calculate the dynamic interval: according to the formula Calculate the current dynamic interval. Wherein, the base interval... =8, proportionality coefficient minimum interval If C(t) = 0 (no dynamic obstacles), then ;like (Extremely complex), then .

[0054] (3) Condition judgment: Check whether the dynamic interval calculated above has been reached. Or whether the performance plateau condition is met (continuous) The average return gain over a period of time is less than If any condition is met, proceed to step S104; otherwise, return to step S102.

[0055] S104: Fitness Assessment

[0056] The parameters of the current PPO network Assign the value to the reference particle in the particle swarm.

[0057] Calculate particle fitness: for each particle in the swarm The position vector is used as a new parameter of the policy network, that is, a temporary policy network is constructed. Run the policy network independently in the simulation environment. In the next round, calculate its average cumulative reward. As the fitness value of the particle This measure aims to fairly evaluate the performance of each set of parameters.

[0058] ;in, This represents the total number of iterations in the simulation environment during the e-th round. This represents the instantaneous reward obtained at time step t in round e.

[0059] S105: Perform GYPSO optimization

[0060] Adaptive parameter adjustment: based on the current GYPSO iteration number Calculate the parameters for this period using the following formula:

[0061]

[0062]

[0063]

[0064] Layering and Updating:

[0065] Based on fitness The particles are sorted into GP (top 4), CP (middle 12), and BP (bottom 4).

[0066] For BP swarm particles, according to the formula Update speed.

[0067] For GP swarm particles, randomly select a peer for comparison; if the peer is superior, then proceed accordingly. Update, or update according to standard PSO.

[0068] For CP group particles, according to the formula Update, in which It is generated by the crossover of elite particles EP and high-quality particles GP according to probability.

[0069] Mixed mutation: Checking the global optimum Has the population not been updated for 5 consecutive generations? If so, perform Cauchy mutation on 20% of the particles in the population. .

[0070] Repeat the GYPSO optimization process iteratively until 20 generations are reached, and record the global optimal solution. .

[0071] S106: Strategy Update and Cycle

[0072] The globally optimal parameters found by GYPSO Policy network assigned to PPO: .

[0073] Return to step S102 and continue the data collection and training process of PPO until the total training cycle reaches the preset value (e.g., 1000 cycles) or the policy performance converges.

[0074] However, it is worth noting that the core of this scheme lies in constructing a dual-loop optimization framework: the inner loop uses the standard PPO algorithm for local policy fine-tuning and value function fitting; the outer loop, when it detects that the performance improvement of the inner loop is slow, starts the improved GYPSO algorithm, treats the policy network parameters as particles to be optimized, performs a global search in the entire parameter space to find a better parameter region, and feeds the optimal result back to the PPO network, thereby breaking the local optimum and guiding the subsequent training direction.

[0075] The specific technical solution of the GYPSO-PPO algorithm includes the following core components:

[0076] 1. GYPSO-PPO Overall Collaborative Framework

[0077] The execution flow of the GYPSO-PPO algorithm is as follows:

[0078] Step S101: Initialize the policy network parameters of the PPO algorithm and value network parameters At the same time, initialize a size of A swarm of particles, each particle's position Represents a set of dimensions and policy network parameters The same vector.

[0079] Step S102: In the environment Running the current policy network Collect trajectory data And calculate the estimated value of the advantage function. .

[0080] Step S103: Based on the collected data, minimize the PPO pruning substitution loss function using stochastic gradient descent. The network parameters are updated K times. The pruning and replacement loss function is defined as follows:

[0081]

[0082] in, For probability ratios, This is the clipping hyperparameter (usually set to 0.2).

[0083] Step S104: Determine whether the GYPSO intervention conditions are met. The conditions are: the current training epoch number is an integer multiple of the preset optimization interval (e.g., 5 epochs), or the average reward improvement rate of the most recent M epochs is lower than a threshold. .

[0084] Step S105: If the condition is met, then change the current PPO strategy parameters. Assign a benchmark particle to the particle swarm and evaluate the fitness of the entire swarm. The fitness function... For particles The average expected cumulative reward of the strategy represented in the environment .

[0085] Step S106: Execute the Adaptive Hierarchical Particle Swarm Optimization (GYPSO) algorithm, iteratively optimize for T generations, and find the global optimum. This step involves the core technologies described in points 2-4 below.

[0086] Step S107: Calculate the globally optimal parameters found by GYPSO. The policy network assigned back to PPO, i.e., .

[0087] Step S108: Repeat steps S102 to S107 until the preset total number of training rounds is reached or the performance converges.

[0088] 2. Adaptive parameter adjustment strategy

[0089] In the iteration process of GYPSO, inertia weight and learning factors , Based on the current iteration number Dynamic adjustment, the adjustment strategy is as follows:

[0090] Inertia weight Used to balance global exploration and local development. Its value ranges from... linearly decreasing to :

[0091]

[0092] Individual learning factors : Controlling particles to their own historical best The learning step size. Its value ranges from... linearly decreasing to :

[0093]

[0094] Social learning factors Controlling particles toward the swarm's historical optimum The learning step size. Its value ranges from... linearly increasing to :

[0095]

[0096] This design allows the algorithm to emphasize exploration in its early stages. and (relatively large), emphasizing utilization and convergence in the later stages ( and Get smaller (Get bigger).

[0097] 3. Hierarchical particle update strategy

[0098] In each generation of GYPSO optimization, the particle swarm is divided into three subgroups based on fitness values, and a differentiated update strategy is adopted:

[0099] Low-mass particle swarm (BP, bottom 20%): Its historical experience value is low, therefore it is discarded during updates. Particles directly selected from the high-quality particle swarm (GP) based on probability Learning. Its speed update formula is:

[0100]

[0101] in Indicates the first The position of a particle is randomly selected from a high-quality particle swarm; ω represents the inertia coefficient. This represents the current position of the i-th particle in the search space at the t-th iteration.

[0102] High-quality particle swarm optimization (GP, top 20%): Employs competitive learning. For particles... Randomly select another particle within the swarm .like Then it will locally develop in the direction of the difference between itself and the random particle:

[0103]

[0104] in, This represents the position of the i-th particle in the high-quality particle swarm at the t-th iteration. This represents the position of a randomly selected particle other than particle i within the same high-quality particle swarm.

[0105] Otherwise, revert to the standard PSO update formula to maintain diversity.

[0106] Ordinary Particle Swarm Optimization (CP, middle 60%): Its updates incorporate the global optimum ( or ) and elite particles ( Information. Hybrid guide samples are generated through random crossover. :

[0107]

[0108] A random number between [0,1]; It is the position of the elite particle swarm in the d-th dimension. It is the position of the high-quality particle swarm in the d-th dimension.

[0109] Its speed update formula is:

[0110]

[0111] in, This represents the component of the historical best position of the i-th particle in the d-th dimension at the t-th iteration. This represents the current position of the i-th particle in the d-th dimension at the t-th iteration. It represents the position of the cross-particle in the d-dimensional dimension at the t-th iteration. These cross-particles are generated through random cross-operations.

[0112] 4. Mixed mutation mechanism

[0113] To prevent premature convergence of the population, the Cauchy mutation operator is introduced. When the global optimum of the population is detected... continuous If the generation is not updated, the population is determined to be stagnant, and then mutation is performed on the particles with a certain probability:

[0114]

[0115] in, These are standard Cauchy distribution random numbers. This is the scaling factor. It refers to the new position a particle occupies in the next generation after undergoing a random mutation. This is the position before the mutation. The long tail of the Cauchy distribution allows it to effectively generate large jumps, helping particles escape local optima.

Claims

1. A hybrid optimization algorithm for collaborative scheduling and dynamic obstacle avoidance in robot swarms, characterized in that, Includes the following steps: S1. Initialize the main policy network of the near-end policy optimization (PPO) algorithm. and value network and initialize a containing A particle swarm of 100 particles, where the position vector of each particle represents a set of parameter values ​​for a PPO policy network. S2, in the environment The current main policy network is running in the middle. Collect state, action, and reward sequence data, and calculate the advantage function estimate; S3. Using the collected data, perform K rounds of optimization on the PPO pruning replacement objective function using stochastic gradient descent, and update the main policy network parameters. and value network parameters ; S4. Determine if the GYPSO intervention conditions are met: If met, proceed to step S5; otherwise, return to step S2 to continue the next round of PPO training. S5. Set the parameters of the current PPO master policy network. Assign a value to a specified particle in the particle swarm, and calculate the fitness value of each particle based on the performance of the strategy it currently represents in the environment. S6. Execute the adaptive hierarchical particle swarm optimization (GYPSO) algorithm: Particles are stratified according to their fitness values, and different velocity and position update strategies are adopted for particles at different levels. Simultaneously, the algorithm parameters are adaptively adjusted based on the population evolutionary state, and a hybrid mutation operation is performed. The algorithm is iteratively optimized for T generations to find the maximum number of iterations. Where T is the maximum number of iterations; S7. Find the maximum number of iterations found by GYPSO. The parameter vector represented is assigned to the main policy network of PPO. This completes the global optimization and update of the strategy parameters; S8. Return to step S2 and continue the PPO data collection and training process until the preset total number of training rounds or performance threshold is reached.

2. The algorithm according to claim 1, characterized in that, The adaptive parameter adjustment strategy in step S6 specifically includes: inertia weight. Individual learning factors Social learning factors All vary with the number of iterations Dynamic changes; inertial weights Used to balance the global exploration and local exploitation capabilities of the algorithm, its value ranges from the maximum value. Decreasing linearly to the minimum value The calculation formula is: Individual learning factor This is used to control how much a particle follows its historical best position, and its value ranges from a maximum value. Decreasing linearly to the minimum value The calculation formula is: Social learning factor This is used to control how much a particle follows the group's historical best position; its value ranges from a minimum. linearly increasing to the maximum value The calculation formula is: This parameter setting allows the algorithm to focus on global exploration in the early stages and on detailed local development in the later stages.

3. The algorithm according to claim 1, characterized in that, The hierarchical particle update strategy in step S6 specifically includes: at the beginning of each generation of optimization, all particles are sorted in ascending order according to their fitness values, the top 20% of particles are classified into the high-quality particle group (GP), the middle 60% into the ordinary particle group (CP), and the bottom 20% into the low-quality particle group (BP); for particles in the low-quality particle group (BP), their updates no longer refer to their own historical best values. Instead, it learns randomly from a particle in the high-quality particle swarm GP, and its velocity update formula is: ,in Indicates the first The position of a particle is randomly selected from a high-quality particle swarm; ω represents the inertia coefficient. This represents the current position of the i-th particle in the search space at the t-th iteration.

4. The algorithm according to claim 3, characterized in that, In step S6, a competitive learning strategy is adopted for particles in the high-quality particle swarm GP: for the first particle in GP... One particle, randomly selected from GP, is different from... particles Compare the fitness of the two; if the particles If its fitness is better, then it follows the formula An update is performed, which involves moving in the direction of the difference between its own position and the random particle's position to enhance its local exploitation capabilities in high-quality regions; among which, This represents the position of the i-th particle in the high-quality particle swarm at the t-th iteration. This represents the position of a randomly selected particle other than particle i within the same high-quality particle swarm. If particles If the fitness is better, then the particle Return to the standard PSO update formula To maintain population diversity and avoid overcrowding; among them, , They are two random numbers between 0 and 1. The best historical position of the i-th particle at the t-th iteration. This represents the global optimal value at the t-th iteration.

5. The algorithm according to claim 3, characterized in that, In step S6, the update strategy for particles in the ordinary particle swarm CP integrates information from the global optimum and elite particles; an elite particle swarm EP is introduced, which consists of particles that exhibit continuous optimization potential during evolution; when ordinary particles update their velocity, their social learning component no longer only aims at the global optimum. Instead, it moves towards a hybrid sample generated by random crossover operations between the global optimal solution and elite particles. The speed update formula is: ;in, This represents the component of the historical best position of the i-th particle in the d-th dimension at the t-th iteration. This represents the current position of the i-th particle in the d-th dimension at the t-th iteration. It represents the position of the cross-particle in the d-dimensional dimension at the t-th iteration. These cross-particles are generated through random cross-operations. Mixed samples Each dimension The corresponding dimension is randomly selected from the elite particle EP or the high-quality particle GP (i.e., the population containing the global optimal solution) based on probability. , A random number between [0,1]; It is the position of the elite particle swarm in the d-th dimension. It is the position of the high-quality particle swarm in the d-th dimension.

6. The algorithm according to claim 1, characterized in that, The hybrid mutation mechanism in step S6 specifically involves continuously monitoring the global optimal solution during the GYPSO iteration process. Update status; if If no improvement is achieved after K consecutive generations, the population is considered to be trapped in a local optimum. In this case, a Cauchy mutation operation is applied to the particles in the current population with a certain probability. The mutation formula is: ,in Random numbers generated from a standard Cauchy distribution. This is the scaling factor; It refers to the new position a particle occupies in the next generation after undergoing a random mutation. It is the position before the mutation.

7. The algorithm according to claim 1, characterized in that, The conditions for GYPSO intervention in step S4 are as follows: (1) Performance plateau judgment: during continuous Within a PPO training cycle, the average reward gain of the policy in the test environment is less than a preset threshold. ; (2) Dynamic Interval Intervention: The current cumulative number of PPO training cycles reaches the dynamically calculated intervention interval. If any condition is met, proceed to step S5. Otherwise, return to step S2 to continue the next round of PPO training; Wherein, the intervention interval From the basic interval Environmental complexity index The calculation is dynamically derived, and the formula is as follows: In the formula This is the proportionality coefficient. This is a preset minimum interval threshold used to prevent excessive intervention.

8. The algorithm according to claim 1, characterized in that, The objective function optimized by GYPSO in step S6 is the objective that the PPO algorithm aims to maximize, and its core is to prune and replace the objective function. For a parameter particle Its fitness value By running the policy network it represents in the environment And evaluate it by calculating its expected cumulative reward, or by calculating it in the current PPO's experience data buffer with parameters. The policy network constructed corresponds to The value is used for approximate evaluation; this design ensures that the optimization direction of GYPSO is completely consistent with the ultimate goal of PPO.

9. The algorithm according to claim 7, characterized in that, The environmental complexity index The calculation is based on one or more of the following factors: the number of dynamic obstacles perceived per unit time, the average congestion level in path planning, and the length of the task queue.

Images