Enhanced particle swarm layout optimization method for base station load

Abstract

The invention relates to a base station load-oriented enhanced particle swarm layout optimization method. Comprising the following steps: step 1, an environment design stage; step 2, a target function construction stage; step 3, an optimization design stage based on an EMEPSO algorithm; step 4, an iterative optimization stage; and the problems of signal attenuation, blind areas and load imbalance in a complex environment can be solved. According to the invention, a feasible solution is provided for optimization of the layout of the base station, and further development of an intelligent base station technology is promoted.

Description

Technical Field

[0001] The present invention relates to an enhanced particle swarm layout optimization method for base station load, belonging to the fields of evolutionary computation and communication technology. Background Art

[0002] With the continuous development and popularization of mobile communication technology, users' demand for high-speed and stable networks continues to grow. Especially under the promotion of 5G and future 6G communication technologies, the requirements for network coverage, capacity, data transmission rate and other performances are getting higher and higher. In this context, the rationality and optimization of base station layout play a crucial role in wireless communication networks. Traditional base station layout mostly adopts empirical methods, which often cannot fully consider various factors in complex environments, such as signal attenuation, load limitation, regional demand differences, etc., resulting in problems such as signal overload, blind spots or uneven resource allocation, thus affecting the overall performance of the network and user experience.

[0003] In practical applications, base station deployment faces various challenges. First of all, signal attenuation is a major problem in base station layout optimization. Especially in areas with severe multipath propagation such as urban complex environments, indoors or underground, the signal attenuation phenomenon is particularly prominent, resulting in incomplete signal coverage and affecting network performance. Secondly, there are large differences in user demands in different regions. Some regions may face problems of overloading or resource waste, while other regions may be in signal blind spots and cannot effectively access the network. Finally, the dynamic allocation and optimization of base station resources are also a major challenge. How to avoid overload or inefficient use through reasonable resource allocation is the key to improving network service quality.

[0004] Currently, in order to improve the efficiency of base station layout, many studies and practices attempt to use optimization algorithms to reasonably plan base station layout. Common optimization methods include genetic algorithms, simulated annealing algorithms, particle swarm optimization algorithms, etc. These methods search for global optimal solutions by simulating the evolutionary process of nature. However, although the traditional particle swarm optimization algorithm has good global search ability in the solution process, it still has defects such as being easily trapped in local optimal solutions and premature convergence. These problems are particularly prominent in complex base station layout optimization, restricting the improvement of optimization effects.

[0005] To solve the above problems, scholars have tried to combine reinforcement learning, adaptive mechanisms and various search strategies to improve the effect of the particle swarm optimization algorithm. For example, the reinforcement learning mechanism can dynamically adjust the search strategy according to environmental changes, while the adaptive mechanism helps to adjust optimization parameters according to the complexity of the current problem to improve the robustness and adaptability of the algorithm. At the same time, introducing a random restart strategy can also effectively avoid local optimal traps in the search process of the algorithm and improve the overall optimization effect. Summary of the Invention

[0006] The present invention provides an enhanced particle swarm layout optimization method for base station load to solve the problem that it is difficult to achieve efficient coverage and uniform signal distribution in the existing base station layout. The present invention proposes an enhanced particle swarm (EMPSO) layout optimization method for base station load, which combines an evolutionary drive strategy and a memory backtracking mechanism to improve the global search ability while taking into account the local optimization accuracy, thereby significantly improving the signal coverage rate and coverage uniformity.

[0007] An enhanced particle swarm layout optimization method for base station load includes the following steps:

[0008] Step 1: Environment design stage: Determine the location distribution of hot spots, which includes fixed-position type and random-position type; among them, the positions of hot spots of the fixed-position type are manually set in advance, and the positions of hot spots of the random-position type are randomly set before each iteration of the algorithm to ensure the difference in each run;

[0009] Step 2: Objective function construction stage: Design a comprehensive objective function that integrates three factors: signal coverage rate target, load balance target, and signal coverage uniformity target. These objective functions will be comprehensively optimized according to weights to form a globally optimal layout plan;

[0010] The signal coverage rate target ensures that in the optimized layout, the signal strength of users in all areas meets the minimum requirements to avoid blind spots; the load balance target adjusts the base station position and load capacity to avoid overloading or being too idle for individual base stations; while the signal uniformity target ensures that there are no areas with too strong or too weak signal coverage, thereby improving the overall service quality. Finally, these objective functions will be comprehensively optimized according to certain weights to form a globally optimal layout plan.

[0011] Step 3: Optimization design stage based on the EMEPSO algorithm: First, initialize the particle swarm. Each particle represents a potential base station layout plan, its position corresponds to the position of the base station, and the velocity represents the amplitude of layout adjustment. Each layout plan is evaluated through the objective function to calculate its fitness value, and then the velocity and position of the particle are adjusted through the particle swarm update algorithm;

[0012] Furthermore, the particle swarm is not only guided by the global optimal solution but also affected by the local optimal solution. To enhance the global search ability of the algorithm, the present invention combines a reinforcement learning mechanism to dynamically adjust the behavior of the particles, guides the particles to search in a better direction through a reward mechanism, and at the same time avoids the particle swarm falling into the local optimal solution. To further improve the search efficiency, the algorithm randomly restarts the positions of some particles within a specific period to broaden the search range and improve the global optimization effect.

[0013] Step 4: Iterative Optimization Phase: The particle swarm undergoes multiple rounds of iteration. In each round of iteration, the fitness of all particles is evaluated, and the velocity and position of the particles are updated according to the evaluation results. As the iteration progresses, the particle swarm gradually converges to the optimal solution, and finally the optimal base station layout plan is output; the convergence criterion is judged according to the set precision requirement and the maximum number of iterations to ensure that the algorithm finds the optimal layout plan within a limited time.

[0014] Step 5: Output Optimization Result Phase: The model generates a detailed base station deployment plan according to the optimized layout plan, including the specific location, coverage radius, load capacity, and signal strength distribution information of each base station. This optimization plan effectively solves problems such as insufficient signal coverage and uneven load, and is particularly suitable for applications in high-density and complex environments.

[0015] A base station layout optimization method, wherein the method calculates the signal coverage rate of each layout plan through the inclusion-exclusion theorem formula (10), and compares the effective coverage areas of each plan during the iteration process;

[0016] (10)

[0017] Wherein, A represents the coverage area of each base station.

[0018] Based on the matrix particle swarm, the base station layout optimization method proposes an evolutionary space domain division strategy. The core of this strategy is that as the algorithm iteration process deepens, the search space of the particles is dynamically adjusted and shrunk, so that it focuses on the potential optimal solution area. This process not only promotes the exploration of particles at a finer scale, but also significantly improves the convergence speed and local search accuracy of the algorithm. To further enhance the balance ability between the global and local searches of particles in a complex fitness environment and improve its navigation accuracy, a particle navigation strategy driven by reinforcement learning is introduced. This strategy integrates the powerful learning ability of deep reinforcement learning, enabling particles to continuously learn and optimize their search behavior according to environmental feedback. Through a large number of iterative trainings, the reinforcement learning model can intelligently identify the optimal search strategy in different fitness environments, thereby guiding the particles to flexibly switch between global search and local fine search in a more efficient manner. The advantages of this algorithm can be specifically summarized as follows:

[0019] Enhanced Global and Local Search Balance: Introducing a particle navigation strategy driven by reinforcement learning effectively balances the needs of global search and local search. Reinforcement learning can dynamically adjust the search strategy of particles according to environmental feedback, optimize the search behavior, and avoid problems such as excessive bias in local search or failure of global search in traditional algorithms.

[0020] Improving algorithm efficiency: By adopting matrix calculation methods, the calculation speed is accelerated. Meanwhile, combining the particle navigation strategy with reinforcement learning enhances the overall efficiency of the particle swarm algorithm, enabling it to have stronger solution-solving capabilities in complex fitness environments, quickly approaching the optimal solution, and reducing ineffective or repetitive calculation processes.

[0021] Intelligence and self-adaptability: Through a large number of iterative trainings, the reinforcement learning model gradually learns and optimizes the search behavior of particles, enabling particles to flexibly respond to various changes in complex environments. This method not only enables the algorithm to adapt to different optimization problems but also allows it to adjust the search strategy according to real-time feedback in various environments, enhancing the intelligence and self-adaptability of the algorithm.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0023] The present invention proposes an improved particle swarm optimization algorithm integrating reinforcement learning and a random restart mechanism, enhancing the adaptability and stability of the algorithm by dynamically adjusting the particle search strategy;

[0024] The present invention comprehensively considers signal attenuation, load balance, and coverage uniformity to construct a comprehensive objective function, ensuring the efficiency and fairness of the optimization results;

[0025] Aiming at the 6G base station layout problem in complex environments, the present invention designs multiple experimental scenarios for verification. The results show that the method of the present invention is significantly superior to traditional methods in terms of signal coverage rate and optimization efficiency.

[0026] From the perspective of model design, the present invention considers various influencing and restrictive factors such as signal attenuation, load balance, and coverage uniformity, realizes a base station communication service closer to real-world requirements, and ensures the signal requirements of hotspots and users.

[0027] In terms of the algorithm solution method, the present invention proposes a particle swarm optimization algorithm integrating evolutionary drive and memory backtracking strategy, which is characterized by high computational efficiency and intelligence, and is helpful for base station communication in hotspots on a large scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0029] Figure 1 It is a signal strength distribution diagram of a single base station in the embodiment of the enhanced particle swarm layout optimization method for base station load of the present application.

[0030] Figure 2 This is a diagram of the reinforcement learning-driven solution in the embodiment of the enhanced particle swarm layout optimization method for base station load in this application.

[0031] Figure 3 This is the EMPSO base station planning flowchart in the embodiment of the enhanced particle swarm layout optimization method for base station load in this application.

[0032] Figure 4 This is the signal coverage effect diagram of the target area in the embodiment of the enhanced particle swarm layout optimization method for base station load in this application.

[0033] Figure 5 : This is the convergence curve of the objective function during the algorithm iteration under random hot spot positions in the embodiment of the enhanced particle swarm layout optimization method for base station load in this application.

[0034] Figure 6 This is the box plot of multiple runs of the algorithm under random hot spot positions in the embodiment of the enhanced particle swarm layout optimization method for base station load in this application. Detailed implementation manners

[0035] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0036] An enhanced particle swarm layout optimization method for base station load includes the following steps:

[0037] Step 1: Environment design stage: Determine the position distribution of hot spot areas, and the position distribution includes fixed position type and random position type; among them, the positions of hot spot areas of the fixed position type are manually set in advance, and the positions of hot spot areas of the random position type are randomly set before each iteration of the algorithm to ensure the difference in each run;

[0038] Step 2: Objective function construction stage: Design a comprehensive objective function that integrates three major factors: signal coverage rate target, load balance target, and signal coverage uniformity target. These objective functions will be comprehensively optimized according to weights to form a globally optimal layout plan;

[0039] Step 3: Optimization Design Phase Based on the EMEPSO Algorithm: First, initialize the particle swarm. Each particle represents a potential base station layout plan, its position corresponding to the location of the base station, and its velocity representing the amplitude of layout adjustment. Each layout plan is evaluated through the objective function to calculate its fitness value, and then the velocity and position of the particles are adjusted through the particle swarm update algorithm;

[0040] Step 4: Iterative Optimization Phase: The particle swarm will go through multiple rounds of iteration. In each round of iteration, the fitness of all particles is evaluated, and the velocity and position of the particles are updated according to the evaluation results. As the iteration progresses, the particle swarm gradually converges to the optimal solution, and finally outputs the optimal base station layout plan;

[0041] Step 5: Output Optimization Result Phase: The model will generate a detailed base station deployment plan according to the optimized layout plan, including the specific location, coverage radius, load capacity, and signal strength distribution information of each base station.

[0042] Base station layout optimization method, wherein the method calculates the signal coverage rate of each layout plan through the inclusion-exclusion theorem formula (10), and compares the effective coverage areas of each plan during the iteration process;

[0043] (10)

[0044] Wherein, A represents the coverage area of each base station.

[0045] The construction of the objective function in the second step specifically includes:

[0046] The objective function comprehensively considers the signal coverage area, the signal demand of the target area, and the load capacity of the base station. The specific form is as follows:

[0047] (1)

[0048] Wherein, f a represents the effective coverage area, f b represents the penalty term for the hotspot area not being fully covered, f c represents the penalty term for not reaching the signal strength of the hotspot area, f d represents the penalty term for exceeding the load capacity.

[0049] The key penalty terms in the objective function include:

[0050] Whether the base station layout achieves full coverage of the hotspot area. For the plans that fail to achieve full coverage, the corresponding penalty terms are introduced;

[0051] (2)

[0052] Among them, M is the total number of hot spots, N is the total number of base stations, BS i represents the i-th base station, P j represents the j-th hot spot area, R i represents the radius of the i-th base station;

[0053] Ensure that the base station layout meets the signal demand intensity of the hot spot area. If not, impose corresponding penalties;

[0054] (3)

[0055] Among them, S gi,j represents the signal strength from base station i to hot spot area j;

[0056] Control the total amount of base station signal output to avoid exceeding the maximum signal load capacity. Once exceeded, impose corresponding penalties;

[0057] (4)

[0058] Among them, L i represents the maximum signal load of each base station.

[0059] The specific steps of step three include:

[0060] Evolution-driven spatial domain division strategy: Dynamically adjust the search space size, and gradually reduce the sub-domain range through formula (5),

[0061] (5)

[0062] Among them, represents the starting size of area division, is the minimum scale of area division; represents the reference division coefficient, set as a constant; represents the upper limit value of the spatial range; is the attenuation factor, used to control the attenuation speed, usually set to 0.01; t is the current generation;

[0063] Reinforcement learning-driven particle navigation strategy: Introduce the Q-Learning algorithm, select the optimal learning action according to the particle state, and update the Q table through formulas (6)-(7).

[0064] (6)

[0065] (7)

[0066] Among them, the left side of the expression Q(S, Action) represents the new S value after executing the action Action in the Q state; the right side Q(S, Action) is the original Q value; R is the immediate reward obtained from the environment after executing the action Action; represents the maximum S' value that can be obtained by selecting the optimal action among all possible actions a' in the next state Q ; is the learning rate, a parameter between 0 and 1, which determines the degree to which new information covers old information. The closer it is to 1, the faster the learning speed. γ is the discount factor, also a parameter between 0 and 1, which determines the importance of future rewards for the current decision. γ The closer it is to 1, the greater the impact of future rewards on the current decision, indicating that the current pays more attention to long-term rewards; conversely, if γ is close to 0, it means that the current values immediate rewards more;

[0067] Random restart and historical memory backtracking mechanism: Record the historical optimal solution, randomly restart the particle with the worst performance, and use the excellent solutions in the historical memory to guide the particle search. Through formula (8), the algorithm can more effectively jump out of the trap of local optimum, readjust the search direction, and then enhance the global search ability.

[0068] (8)

[0069] (9)

[0070] Among them, is any solution in the set of historical excellent solutions, where and respectively represent the maximum and minimum perturbation amounts under the current exploration space scale. represents the maximum number of iterations, indicates that this is the i-th iteration.

[0071] Matrix particle swarm: To further improve the execution efficiency of the algorithm, this paper introduces the matrix particle swarm algorithm. That is, assume there is a population of N individuals to solve a D-dimensional problem. Under this framework, assume there is a population of N individuals used to solve a D-dimensional optimization problem. It reduces the loop calculation within the population and speeds up the calculation speed of the algorithm.

[0072] Enhanced particle swarm algorithm process

[0073] 1. Initialize the particle swarm, set the search space and parameters, and start iteration;

[0074] 2. Calculate the size of the current sub-domain, and then determine the search area of the particles.

[0075] 3. Calculate the central point position of the optimal solution in the particle swarm, and use it as a reference point to determine the positions of each particle and the whole swarm.

[0076] 4. Detect whether each particle is in the space of the current global optimal solution.

[0077] 5. Determine the state of the particle according to the current position, velocity of the particle and environmental feedback.

[0078] 6. Select an appropriate learning action according to the current state of the particle. This selection is based on the Q-value (behavior value) in reinforcement learning.

[0079] 7. Update the velocity of the particle according to the selected action.

[0080] 8. Update the position of the particle and move according to the new velocity.

[0081] 9. Calculate the fitness value of the particle according to its current position. The fitness value is used to measure the quality of the solution. Calculate the reward value: Calculate the reward value of the particle to reflect the contribution of the current action to the overall optimization.

[0082] 10. Update the Q-table and adjust the strategy for future decisions to make more effective choices in the next step.

[0083] 11. Update the historical optimal solution of the swarm and the personal historical optimal solution according to the performance of the current particle.

[0084] Based on the matrix particle swarm, the base station layout optimization method proposes an evolutionary space sub-domain strategy. The core of this strategy lies in dynamically adjusting and shrinking the search space of the particles as the algorithm iteration process progresses, so that it focuses on the potential optimal solution area. This process not only promotes the exploration of particles at a finer scale, but also significantly improves the convergence speed and local search accuracy of the algorithm. To further enhance the balance ability between the global and local searches of particles in a complex fitness environment and improve its navigation accuracy, a particle navigation strategy driven by reinforcement learning is introduced. This strategy integrates the powerful learning ability of deep reinforcement learning, enabling particles to continuously learn and optimize their search behaviors according to environmental feedback. Through a large number of iterative trainings, the reinforcement learning model can intelligently identify the optimal search strategies in different fitness environments, thereby guiding the particles to flexibly switch between global search and local fine search in a more efficient manner. The advantages of this algorithm can be specifically summarized as follows:

[0085] Enhanced global and local search balance: Introduce a particle navigation strategy driven by reinforcement learning, effectively balancing the needs of global search and local search. Reinforcement learning can dynamically adjust the search strategy of particles according to environmental feedback, optimize the search behavior, and avoid the problems of excessive bias in local search or failure of global search in traditional algorithms.

[0086] Improve algorithm efficiency: Adopt matrix calculation to speed up the calculation speed. At the same time, combine the particle navigation strategy with reinforcement learning to improve the overall efficiency of the particle swarm algorithm. It has stronger problem-solving ability in complex fitness environments, can quickly approach the optimal solution, and reduces the ineffective or repetitive calculation process.

[0087] Intelligence and self-adaptability: Through a large number of iterative trainings, the reinforcement learning model gradually learns and optimizes the search behavior of particles, enabling particles to flexibly respond to various changes in complex environments. This method enables the algorithm not only to adapt to different optimization problems, but also to adjust the search strategy according to real-time feedback in various environments, enhancing the intelligence and self-adaptability of the algorithm.

[0088] Dynamic programming process for base station layout based on intelligent adjustment and complex environment optimization:

[0089] First, it is necessary to determine the location distribution of hot spots, which includes two types: fixed location type and random location type. The location of hot spots of the fixed location type is set manually in advance, while the location of hot spots of the random location type is obtained through random calculation in each algorithm iteration to ensure the difference in each run. The number of particles in the algorithm represents the number of different base station layout schemes. Then, initialize the base station locations in each base station layout scheme. During the iteration process, each scheme will gradually optimize its base station location by absorbing and learning the schemes with better performance in the layout scheme set and selecting different adjustment schemes according to its own state. Through multiple iterations, it gradually approaches the best deployment scheme under the base station signal model.

[0090] During the optimization process, a series of evaluation criteria are adopted to ensure the effectiveness of base station deployment. Specifically, evaluate whether the base station layout achieves full coverage of the hot spot area. For the schemes that fail to achieve full coverage, corresponding penalty terms will be introduced. At the same time, ensure that the base station layout can meet the signal demand intensity of the hot spot area. If not, the same penalty will be imposed on this scheme. In addition, control the total amount of base station signal output to avoid exceeding the maximum signal load capacity. Once exceeded, corresponding penalties will also be introduced. To quantify the quality of base station deployment, calculate the signal coverage rate of each scheme. During the iteration process, by comparing the effective coverage areas of each scheme and comprehensively considering each evaluation index, finally select the base station layout solution with the best overall performance.

[0091] The present invention proposes an enhanced particle swarm layout optimization method for base station load. The optimization method includes analyzing signal attenuation and load distribution for the base station layout, and proposing an objective function applicable to complex environments; modeling the signal attenuation phenomenon according to the objective function, dividing the signal coverage area and optimizing it; designing a particle swarm optimization algorithm based on a reinforcement learning adaptive mechanism, and introducing a random restart strategy to avoid the local optimal solution and premature convergence problems of traditional algorithms; simulating the base station layout optimization for the experimental area, and collecting the signal coverage rate and load balancing situation; inputting the optimized base station layout scheme into the EMEPSO algorithm for adjustment, and achieving global signal coverage and optimal resource allocation. This method can effectively improve the signal coverage rate and resource allocation efficiency of the base station layout, has high optimization accuracy and speed, and can cope with signal attenuation, blind spots and load imbalance problems in complex environments. The present invention provides a practical solution for the optimization of the base station layout, which helps to promote the further development of intelligent base station technology.

[0092] The above embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, without departing from the principle and spirit of the present invention, various changes, modifications, substitutions and variations of these embodiments still fall within the protection scope of the present invention.

Claims

1. An enhanced particle swarm layout optimization method for base station load, characterized by: The following steps are involved: Step 1: Environmental design stage: determine the location distribution of hotspot areas, which include fixed location type and random location type; the location of the hotspot area of ​​the fixed location type is manually set in advance, and the location of the hotspot area of ​​the random location type is randomly set before each iteration of the algorithm to ensure the difference of each run; Step 2: Objective function construction phase: A comprehensive objective function is designed, integrating the three factors of signal coverage target, load balancing target and signal coverage uniformity target. These objective functions will be comprehensively optimized according to the weights to form a global optimal layout solution; Step 3: Optimization design phase based on EMEPSO algorithm: First, initialize the particle swarm. Each particle represents a potential base station layout plan. Its position corresponds to the position of the base station, and its speed represents the amplitude of the layout adjustment. Each layout plan is evaluated through the objective function, and its fitness value is calculated. Then, the particle speed and position are adjusted through the particle swarm update algorithm. Step 4: Iterative optimization phase: The particle swarm will go through multiple rounds of iterations. Each round of iteration will evaluate the fitness of all particles and update the particle speed and position based on the evaluation results. As the iterations proceed, the particle swarm gradually converges to the optimal solution and finally outputs the optimal base station layout plan. Step 5: Output optimization results: The model will generate a detailed base station deployment plan based on the optimized layout plan, including the specific location, coverage radius, load capacity and signal strength distribution information of each base station.

2. The enhanced particle swarm layout optimization method for base station load according to claim 1, characterized in that: The step 2 of constructing the objective function specifically includes: The objective function comprehensively considers the signal coverage area, the signal demand of the target area, and the load capacity of the base station. The specific form is as follows: （1） in, f a Expressed as the effective coverage area, f b Indicates the penalty item for not fully covering the hotspot area, f c Indicates the penalty item for not reaching the signal strength of the hotspot area. f d Represents the penalty term for exceeding the load capacity.

3. The enhanced particle swarm layout optimization method for base station load according to claim 2 is characterized in that: The key penalty terms in the objective function include: For schemes that fail to achieve full coverage, corresponding penalties are introduced; （2） in, M is the total number of hotspot areas, N is the total number of base stations, BS i Representative i Base stations, P j represents the jth hotspot area, R i Representative i The radius of the base station; Ensure that the base station layout meets the signal strength requirements of hot spots. If not, impose corresponding penalties; （3） in, S gi,j Indicates base station i To hot spots j Signal strength; Control the total amount of base station signal output to avoid exceeding the maximum signal load capacity. Once exceeded, impose corresponding penalties; （4） in, L i Represents the maximum signal load of each base station.

4. The enhanced particle swarm layout optimization method for base station load according to claim 1, characterized in that: The step three specifically includes: Evolution-driven spatial domain division strategy: Dynamically adjust the search space size and gradually reduce the subdomain range through formula (5). （5） in, Indicates the starting size of the region segmentation. is the minimum scale for region segmentation; represents the benchmark split coefficient, which is set to a constant; Indicates the upper limit of the spatial range; is the attenuation factor, used to control the attenuation speed, usually set to 0.01; t is the current algebra; Reinforcement learning driven particle navigation strategy: Introducing the Q-Learning algorithm, select the optimal learning action according to the particle state, and update it through formulas (6)-(7) Q surface; （6） （7） Among them, the left side of the expression Q(S,Action) Indicates status S Next, after executing the action, Q Value; right Q (S,Action) It is the original Q value; R It is the immediate reward obtained from the environment after performing the action; Indicates the next state S' By selecting all possible actions a' The maximum value that can be obtained by the optimal action in Q value; is the learning rate, which is a parameter between 0 and 1 and determines the extent to which new information covers old information. The closer it is to 1, the faster the learning speed. γ is the discount factor, which is also a parameter between 0 and 1, and determines the importance of future rewards to current decisions. γ The closer it is to 1, the greater the impact of future rewards on current decisions, indicating that long-term rewards are more important at present; conversely, if γ If it is close to 0, it means that immediate rewards are more important at present; Random restart and historical memory backtracking mechanism: record the historical optimal solution, randomly restart the worst-performing particle, and use the excellent solution in the historical memory to guide the particle search. Through formula (8), the algorithm can more effectively jump out of the local optimal trap, readjust the search direction, and thus enhance the global search capability. （8） （9） in, is any solution in the set of historical excellent solutions, where and They represent the maximum and minimum perturbations under the exploration space scale, represents the maximum number of iterations, Indicates that this is the i Iterations.

Images