A near-end policy enhanced ant colony optimization path coverage method

Abstract

The application discloses a near-end strategy enhanced ant colony optimization path coverage method and belongs to the technical field of intelligent path planning. In a grid environment with various complex obstacles and variable scales, through the introduction of a dynamic hybrid decision mechanism, a feedback pheromone updating strategy, a path redundancy suppression method and a double-threshold value constraint of pheromone concentration and other core mechanisms, the overall coverage rate and convergence speed of the path are significantly improved, the repeated access behavior is effectively suppressed, the robustness and adaptability of the algorithm are enhanced, and therefore, the application provides a comprehensive coverage, simple path and rapid response high-quality planning path for applications such as autonomous mobile robots, unmanned inspection systems and intelligent cleaning equipment, and finally realizes efficient and reliable full-path coverage autonomous operation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent path planning, and particularly relates to an ant colony optimization path coverage method with near-end strategy enhancement, and particularly applies to full coverage path planning in a grid environment with obstacles. BACKGROUND

[0002] Currently, mainstream methods in the field of full path coverage planning can be divided into three categories: the first is a bionic swarm intelligence algorithm such as ant colony optimization (ACO), which searches for a feasible path in a large space by simulating the cooperation and pheromone mechanism of biological groups; the second is a reinforcement learning method, such as proximal policy optimization (PPO), which models the coverage task as a Markov decision process and realizes adaptive response to the environment state through policy gradient optimization; and the third is other meta-heuristic algorithms including genetic algorithms and particle swarm optimization, which realize path generation and optimization by means of population evolution or social behavior models.

[0003] However, these methods have obvious limitations: ACO is prone to local optimization, and the diversity decreases in the later stage of convergence, and the generated path has high redundancy; reinforcement learning methods such as PPO have problems such as low sample efficiency, difficulty in exploration-exploitation balance, and long training period, making it difficult to cope with large-scale expansion of state space or dynamic changes in the environment; other meta-heuristic algorithms generally have common defects such as complex operator design, unguaranteed convergence, parameter sensitivity, and early maturation, and have limited coverage performance and insufficient stability in complex obstacle scenarios. SUMMARY

[0004] In view of the above deficiencies in the prior art, the ant colony optimization path coverage method with near-end strategy enhancement provided by the present application systematically solves the key problems of traditional methods in convergence, planning efficiency and environmental adaptability by deeply integrating the global exploration ability of swarm intelligence and the local decision advantage of reinforcement learning.

[0005] In order to achieve the above application purposes, the technical scheme adopted by the present application is as follows: an ant colony optimization path coverage method with near-end strategy enhancement, comprising the following steps:

[0006] S100, model the working environment as a grid space, and initialize the visited set and the taboo table in the path planning process, and construct the observation vector of the individual in the working environment when path planning is performed using proximal policy optimization (PPO);

[0007] S200, perform iterative path planning in the grid space based on a combination of hybrid ant colony optimization (ACO) and proximal policy optimization (PPO);

[0008] In each iteration, the number of unvisited neighborhoods in the current candidate path generation process is calculated to form a heuristic multiplier, and the occurrence frequency of the current generated candidate path is calculated to form a congestion correction term. The heuristic multiplier and the congestion correction term are used as a combined heuristic.

[0009] S300. Based on the combinatorial heuristic, calculate the probability of each candidate action when generating candidate paths in each iteration when using hybrid ant colony optimization (ACO) for global path planning; based on the individual's observation vector, calculate the probability of each candidate action when generating candidate paths in each iteration when using proximal strategy optimization (PPO) for local path planning.

[0010] The probability of candidate actions is adaptively fused as the iteration progresses to generate action decision probabilities. Then, the path is expanded during the iteration process in a way that is either action sampling or greedy, generating all paths and updating the tabu list and visited set simultaneously.

[0011] S400: Based on the path coverage and path redundancy of all paths, perform path redundancy suppression.

[0012] S500, based on the path redundancy suppression results, releases pheromones during the path planning process through elite path weighting and global optimal path enhancement mechanisms, and introduces a feedback adjustment mechanism based on coverage progress to adaptively update the pheromone evaporation rate until the coverage of the generated path meets the set task threshold, thus obtaining a fully covered global optimal path.

[0013] Furthermore, in step S100, during the path planning process using a near-end strategy to optimize PPO, when the first... The individual is in position At that time, the individual's observation vector for:

[0014] ;

[0015] ;

[0016] ;

[0017] ;

[0018] ;

[0019] In the formula, This represents the normalized vector of the current position. This indicates that the binary graph tile vector has been visited. Represents the progress ratio vector. Indicates the number of steps. This indicates that the mask has been accessed. Represents the matrix vectorization operator. This represents the progress fusion coefficient between control coverage and step weighting. express Step coverage, Indicates relative progress in steps. This represents the total dimension of the observation vector. , Indicates the length and width of the rasterized space. Indicates the first The position coordinates of an individual in the rasterized space during a step. It represents the set of discrete states corresponding to the rasterized space.

[0020] Furthermore, in step S200, the heuristic multiplier Represented as:

[0021] ;

[0022] ;

[0023] In the formula, The heuristic value representing the candidate action. Represents the heuristic weighting factor. Represents candidate grid points for individuals during path planning. The number of unvisited feasible neighbors in its 8-neighborhood. Represents grid points Is it in the set of feasible units? Indicator functions, Represents grid points An indicator function indicating whether it has been visited. Representing candidate grid points A certain neighborhood grid point, Representing candidate grid points The set of 8 neighboring grid points;

[0024] The congestion correction item Represented as:

[0025] ;

[0026] ;

[0027] In the formula, Indicates the first The number of paths to grid point 𝑗 included in each iteration. Indicates the crowding suppression weight. Indicates the first The set of paths generated by all individuals during each iteration. This represents the set of raster points after removing duplicates from the path raster point set. This represents the function that takes the maximum value. This indicates the generation of a path index.

[0028] Furthermore, in step S300, when using hybrid ant colony optimization (ACO) for global path planning, the probability of each candidate action when generating candidate paths is expressed as follows:

[0029] ;

[0030] ;

[0031] In the formula, This indicates that when using a hybrid ant colony to optimize ACO, in Step individual from grid point To the candidate next grid point The probability, This represents the scoring function for hybrid ant colony optimization of ACO. Indicates in During step, grid points To the candidate next grid point The intensity of pheromones on the surrounding area This represents a heuristic for combining candidate actions, indicated by a superscript. and These represent the pheromone heuristic factor and the expectation heuristic factor, respectively. Indicates the next candidate grid point Another different candidate next grid point, Indicates the first Time-lapse grid points The set of legal candidate grid points;

[0032] When using a proximal strategy to optimize PPO for local path planning, the probability of each candidate action when generating candidate paths is expressed as follows:

[0033] ;

[0034] In the formula, This indicates that when using a proximal strategy to optimize PPO, in Step individual from grid point To the candidate next grid point The probability, This represents the action probability of the original output of the policy network when using a near-end policy to optimize PPO. Indicates the candidate next grid point Candidate mask, This indicates that when using a proximal strategy to optimize PPO, in the first... Step-by-step policy network for grid points To the candidate next grid point action probability of the action original output, a candidate mask representing candidate next grid points ;

[0035] fusing the probabilities of the candidate actions adaptively with iteration progress to generate action decision probability is represented as:

[0036] ;

[0037] wherein, is a fusion factor changing with iteration progress.

[0038] Further, in the step S400, path coverage rate of the whole paths is:

[0039] ;

[0040] wherein, is coverage rate of the whole paths , is weight of , is length of the whole paths , is weight of ;

[0041] path redundancy rate of the whole paths is:

[0042] ;

[0043] wherein, is path redundancy rate of the whole paths , is total access step number of the whole paths, is de-duplicated visited grid set of the whole paths .

[0044] Further, in the step S500, the method for pheromone release by elite path weighting is:

[0045] According to the path redundancy inhibition result, the current paths are sorted, and the first several paths are selected as elite paths;

[0046] The pheromone is released by weighting according to the elite path sorting result, and the release strength decreases with the size of the elite path sorting, and is normalized by the length of the elite path.

[0047] Further, in the step S500, the historical global optimal path is additionally released pheromone, which is represented as:

[0048] ;

[0049] wherein, denotes the historical global optimal path denotes the pheromone increment added along each edge, denotes the global reinforcement weight, denotes the pheromone release constant, denotes the length of the historical global optimal path.

[0050] Further, the feedback adjustment mechanism based on the coverage progress in the step S500 comprises:

[0051] quantifying the coverage progress based on the path coverage, adjusting the pheromone evaporation coefficient based on the feedback mechanism of the coverage progress, quantifying the progress stagnation based on the coverage increment, and adjusting the pheromone release coefficient based on the coverage progress and the progress stagnation.

[0052] Further, the formula for quantifying the coverage progress based on the path coverage is:

[0053] ;

[0054] ;

[0055] wherein, denotes the progress promotion value, and denote the coverage rates of the first step and the second step, respectively, denotes the visited mask of the candidate grid coordinates, denotes the candidate grid coordinates of the individual in the raster space during the path planning process, denotes the feasible unit set; The formula for adjusting the pheromone evaporation coefficient based on the feedback mechanism of the coverage progress is:

[0056] ;

[0057] ;

[0058] wherein, denotes the pheromone evaporation coefficient, denotes the basic evaporation rate, denotes the adjustment coefficient for controlling the influence of the progress on the evaporation rate;

[0059] The formula for quantifying the progress stagnation based on the coverage increment is:

[0060] ;

[0061] wherein,​​ an indicator of stagnation of progress, an indicator of coverage growth being less than a set threshold a number of consecutive iteration steps;

[0062] adjusting the pheromone release coefficient based on the coverage progress and the progress stagnation information is:

[0063] when the pheromone evaporation coefficient is:

[0064] ;

[0065] when the pheromone evaporation coefficient is:

[0066] ;

[0067] wherein, and respectively represent the weights of and .

[0068] Further, in the process of updating the pheromone, it also includes: applying a lower and upper threshold value to the pheromone matrix obtained by updating each iteration in the path planning process, which is represented as:

[0069] ;

[0070] ;

[0071] wherein, represents the lower threshold value of the pheromone, represents the upper threshold value of the pheromone, represents the initial pheromone, represents the initial level of pheromone release, represents the order of magnitude of the pheromone release per unit length, represents the pheromone release constant, and represents the upper limit of the global ratio of the pheromone.

[0072] The beneficial effects of the present application are:

[0073] The path coverage quality is significantly improved: through the deep integration of the global exploration of ACO and the local optimization ability of PPO, a coverage rate of up to 97.94% is achieved in a complex obstacle grid environment. The introduction of the path redundancy suppression mechanism reduces the redundancy rate to 0%, effectively avoiding invalid repeated visits, making the path more compact and efficient. The coverage-length double objective scoring mechanism ensures the balance between coverage rate and path length, making the generated path comprehensive and efficient.

[0074] Convergence speed and algorithm efficiency optimization: Compared with the traditional ant colony optimization ACO algorithm, the convergence time is reduced by 86%, and high-quality path planning can be completed in only 30.67 seconds in a 10x10 grid environment; the dynamic hybrid decision mechanism realizes the adaptive transition from exploration to utilization, relying on ACO to quickly expand the coverage front in the early stage, and relying on PPO for fine repair in the later stage; the feedback pheromone update intelligently adjusts the evaporation rate according to the coverage progress, accelerating the convergence process.

[0075] Algorithm stability and robustness enhancement: The pheromone double threshold constraint effectively prevents the excessive concentration or evaporation of pheromone concentration, avoiding premature convergence and search stagnation; the elite ranking weighted release and global optimal path secondary reinforcement form a dual experience solidification mechanism of "short-term memory + long-term memory", ensuring stable convergence of the algorithm in complex environments; the group congestion suppression makes the path search evenly distributed in the entire environment, maintaining population diversity and improving adaptability to different environmental layouts.

[0076] Environmental adaptability and practicality: The method of the present application is suitable for grid environments with obstacles and can be widely used in global coverage scenarios such as autonomous inspection robots, warehouse inventory, agricultural plant protection drones, etc. The design of the observation vector enables the PPO strategy to perceive multi-dimensional information such as position, coverage state and progress, reflecting environmental adaptability in local decision-making. In the global coverage task with fixed starting point and no preset endpoint, the algorithm can autonomously explore and efficiently complete coverage, having strong practical value.

[0077] Technical innovation and systematization: For the first time, the swarm intelligence of ant colony optimization ACO and the reinforcement learning of proximal policy PPO are iteratively integrated to adapt to progress, solving the inherent limitations of single algorithms. A complete global coverage path planning framework is proposed, forming a closed loop from environment modeling, decision fusion to evaluation update, systematically solving key problems in path coverage. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1 The proximal policy enhanced ant colony optimization path coverage method flowchart provided by the present application.

[0079] Figure 2 The iterative convergence effect diagram provided by the present application.

[0080] Figure 3 The global optimal path schematic diagram provided by the present application. DETAILED DESCRIPTION

[0081] The specific embodiments of the present application are described below to facilitate the understanding of the present application for those skilled in the art, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

[0082] The embodiment of the present application provides an end-to-end policy enhanced ant colony optimization path coverage method. In a variety of complex obstacles and variable scale grid environment, by introducing dynamic hybrid decision mechanism, feedback pheromone update strategy, path redundancy suppression method and pheromone concentration double threshold constraint, the overall coverage rate and convergence speed of the path are significantly improved, the repeated access behavior is effectively suppressed, the robustness and adaptability of the algorithm are enhanced, and finally the high-quality planning path with comprehensive coverage, simple path and rapid response is provided for autonomous mobile robot, unmanned inspection system, intelligent cleaning equipment and the like, so that efficient and reliable full path coverage autonomous operation is realized.

[0083] Reference Figure 1 The full coverage path planning method comprises the following steps:

[0084] S100, modeling a working environment as a grid space, and initializing a visited set and a taboo table in a path planning process, and constructing an observation vector of an individual in the working environment when path planning is performed by using proximal policy optimization (PPO);

[0085] S200, performing iterative path planning in the grid space based on a combination of ant colony optimization (ACO) and proximal policy optimization (PPO);

[0086] In each iteration, the number of unvisited neighbors in the current candidate path generation process is calculated to form a heuristic multiplier, and the frequency of the current generated candidate path is calculated to form a congestion correction term, and the heuristic multiplier and the congestion correction term are used as a combined heuristic;

[0087] S300, according to the combined heuristic, calculating the probability of each candidate action when generating a candidate path in each iteration when global path planning is performed by using the ant colony optimization (ACO), and according to the observation vector of the individual, calculating the probability of each candidate action when generating a candidate path in each iteration when local path planning is performed by using the proximal policy optimization (PPO);

[0088] The probabilities of the candidate actions are adaptively fused with the iteration progress to generate an action decision probability, and then the path is expanded in the iteration process in the form of action sampling or greediness to generate a full path, and the taboo table and the visited set are updated synchronously;

[0089] S400, based on the path coverage of all paths and the path redundancy rate, path redundancy suppression is carried out;

[0090] S500, based on the path redundancy suppression result, the pheromone in the path planning process is released through the elite path weighting and global optimal path reinforcement mechanism, and a feedback regulation mechanism based on coverage progress is introduced to adaptively update the evaporation rate of the pheromone until the coverage rate of the generated path meets the set task threshold, and a global optimal path with full coverage is obtained.

[0091] In the application, the working environment of the individual to be planned path (autonomous mobile robot, unmanned inspection system, intelligent cleaning equipment, etc.) is modeled as a grid space; specifically, it is represented as a two-dimensional grid, obstacles are represented as a binary matrix, each grid corresponds to a discrete state, an 8-neighborhood connectivity model is used, and four-way and diagonal movement is allowed. In the path planning process, at any time, the "feasible cell set" is used as the search domain, and any action generation and evaluation must satisfy the boundary legality and obstacle avoidance.

[0092] In step S100, assuming that the working environment is a discrete grid with size HxW, the discrete state set is defined as follows:

[0093] ;

[0094] The boundary legality indication function is defined as:

[0095] ;

[0096] The obstacle is represented by a binary matrix , , otherwise it is passable.

[0097] The obstacle set is defined as follows:

[0098] ;

[0099] The feasible cell set (global search domain) is defined as follows:

[0100] ;

[0101] The above working environment modeling process is applicable to indoor robots, warehouse inventory, corridor inspection, etc. Regular space, and can also be extended to agricultural plant protection routes, etc. Semi-regular space.

[0102] On the basis of constructing the grid space, in order to accurately evaluate coverage and redundancy, the "visited set" is defined and initialized, specifically, the access sequence coverage is measured by the proportion of different grids in the feasible unit; the path geometric length is accumulated by 4 / 8 connectivity cost, the axial step length is 1, and the diagonal step length is . This definition is consistent with the real maneuvering cost of the mobile platform, and also provides a basis for the subsequent comprehensive score of the "coverage-length dual target".

[0103] Further, the taboo table is set to record the grid point sequence walked by the current individual in this search process, avoiding meaningless return immediately. At each time, for the current grid point, all candidate next steps that meet the conditions of "not crossing the border, non-obstacle, not in the current taboo table" are generated.

[0104] In step S100, in order to enable the subsequent path planning by adopting the proximal strategy optimization PPO to play a local insight role in the coverage task, when the first individual is at the position , , the observation vector of the individual is constructed as:

[0105] ;

[0106] ;

[0107] ;

[0108] ;

[0109] ;

[0110] In the formula, , represents the current position normalized vector, , represents the visited binary map paving vector, , represents the progress proportion vector, , represents the step number, , represents the visited mask, , represents the matrix vectorization operator, , represents the progress fusion coefficient for controlling the coverage rate and the step number weight, , represents the coverage rate of the th step, , represents the relative step progress, , represents the total dimension of the observation vector, that is, the size of the observation space, , , represents the length and width of the grid space, , represents the position coordinates of the individual in the grid space at the th step, a set of discrete states corresponding to the rasterized space.

[0111] wherein, ; , the visited mask at the grid coordinate , , , .

[0112] The observation vector design strategy provided by the embodiment can be directly aligned with the candidate action set of the hybrid ant colony optimization ACO, and participates in fusion at different iteration stages.

[0113] In step S200, in order to encourage advancing to an area with large coverage gain, the number of unvisited feasible neighbors in the 8-neighborhood of each candidate grid is calculated, and the higher the value, the more coverage potential there is to be expanded centered on the point. This factor is used as a heuristic multiplier in probability evaluation, so as to enlarge the selection opportunity of such candidate. Thus, the heuristic multiplier is obtained is represented as:

[0114] ;

[0115] ;

[0116] In the formula, represents the heuristic value of the candidate action, represents the heuristic weight factor, represents the candidate grid point of the individual in the path planning process the number of unvisited feasible neighbors in the 8-neighborhood thereof, represents an indication function of whether the grid point is in the set of feasible cells, represents an indication function of whether the grid point has been visited, represents a certain neighbor grid point of the candidate grid point represents the 8-neighborhood grid point set of the candidate grid point . The heuristic multiplier provided by the embodiment is different from the selection of the conventional pheromone alone, and the heuristic can significantly improve the coverage growth speed.

[0117] Further, considering the congestion phenomenon that multiple paths squeeze into the same corridor often occurs in group search, the present application counts the frequency of occurrence of a certain candidate node in all paths of the current generation, and constructs a congestion correction term based on the frequency of occurrence, which is represented as:

[0118]

[0119] ​​ ;

[0120] ;

[0121] In the formula, represents the number of paths containing the grid point j in the iteration of step , represents the congestion suppression weight, the greater the weight value, the stronger the punishment, represents the number of paths generated by all individuals at the iteration of step , represents the grid point set after deduplication of the path grid point set, represents the maximum value function, represents the generated path index.

[0122] The congestion correction term calculation method provided in the embodiment effectively solves the problem of group congestion in the traditional hybrid ant colony optimization ACO. The path search can be more evenly distributed in the entire environment, avoiding the concentration of multiple individuals on the same path, and maintaining the diversity and coverage in the search process. This mechanism plays a crucial role in avoiding local convergence of path search and improving global coverage efficiency.

[0123] In step S300, in path planning, two types of probabilities are given for each candidate action: one is from the pheromone of hybrid ant colony optimization ACO-combination heuristic (pheromone reflects historical experience, and heuristic reflects local potential), and the other is from the policy distribution of proximal policy optimization PPO for the current state (reflects learned local preferences and constraint adaptation).

[0124] Specifically, when using hybrid ant colony optimization ACO for global path planning, the probability of each candidate action when generating a candidate path is represented as:

[0125] ;

[0126] ;

[0127] In the formula, represents the probability of an individual moving from grid point to the candidate next grid point at step , represents the scoring function of hybrid ant colony optimization ACO, represents the pheromone intensity on the edge from grid point to the candidate next grid point at step , represents the combination heuristic of the candidate action, and the superscript and respectively represent pheromone heuristic factor and expected heuristic factor, represents another candidate next grid point different from the candidate next grid point represents the set of legal candidate grid points of the grid point at the t-th step.

[0128] When the local path planning is performed by using the proximal policy optimization PPO, the probability of each candidate action when generating a candidate path is represented as:

[0129]

[0130] In the formula, represents the probability of an individual from the grid point to the candidate next grid point at the t-th step when the proximal policy optimization PPO is used, represents the action probability of the original output of the policy network, represents the candidate mask of the candidate next grid point represents the action probability of the original output of the policy network for the action from the grid point to the candidate next grid point at the t-th step when the proximal policy optimization PPO is used, represents the candidate mask of the candidate next grid point

[0131] In the above process, two kinds of action probabilities are calculated respectively, which effectively solves the contradiction between global exploration and local utilization in traditional path planning. In the early stage, the hybrid ant colony optimization ACO is responsible for extensive exploration of the path, and quickly finds a potential feasible path. With the progress, the proximal policy optimization PPO gradually enhances the local optimization ability, and improves the quality of the path in detail, and finally balances between the global and the local. This strategy is particularly suitable for dynamic environments, such as unmanned vehicle inspection, agricultural unmanned plant protection and the like.

[0132] Further, the fusion factor growing with the iteration progress is proposed in the present application, which combines the two kinds of probabilities in a multiplicative manner and normalizes them, so that the global exploration of the hybrid ant colony optimization ACO is more dependent in the early stage, and the local utilization of the proximal policy optimization PPO is gradually enhanced in the later stage.

[0133] Therefore, the probability of the candidate action is adaptively fused with the iteration progress, and the action decision probability is represented as: ​​​​​​​

[0134] ;

[0135] In the formula, represents the fusion factor changing with the iteration progress.

[0136] The adaptive fusion process proposed in this embodiment follows the migration logic from exploration to utilization. In practice, this mechanism enables the algorithm to quickly expand the coverage frontier in the early stage and pay more attention to detail repair and redundancy compression in the later stage.

[0137] After the double-source action probability calculation and iteration progress adaptive fusion, each candidate action has obtained the final selection probability distribution. Based on the distribution, the individual (ant / agent) performs action sampling or greedy selection at the current node, appends the selected result to the path sequence, and synchronously updates the taboo list and the "visited set", ensuring that subsequent decisions can avoid invalid return and correctly count the coverage range.

[0138] This process is executed in parallel for all individuals in the population until any of the following conditions occurs:

[0139] (1) No legal candidate (all surrounding nodes are obstacles or have been visited);

[0140] (2) Coverage growth stagnation (coverage rate has not improved within a preset number of steps).

[0141] Through the above process mechanism, the decision modeling at the probability level transitions to the actual generation at the path level, and a set of candidate path sets is obtained for each generation. These paths not only intuitively reflect the execution effect of the fusion probability, but also provide necessary input samples for subsequent coverage evaluation and redundancy suppression, realizing the closed-loop connection from "decision → action → evaluation".

[0142] In step S400, the comprehensive score is constructed with "coverage rate as positive target and path length as negative target" to evaluate the path coverage rate, and the path coverage rate of all paths is obtained :

[0143] ;

[0144] In the formula, represents the coverage rate of all paths , represents the weight of , represents the length of all paths , represents the weight of .

[0145] Wherein, the coverage rate reflects the task completion degree, and the path length reflects the execution cost, and the two are combined in the form of weighted difference; this evaluation method has interpretability and scalability for different applications (cleaning, inspection, spraying); the score is not fixed to a certain value, but is a relative scale for "ranking and updating", which can be used for subsequent selection of contemporary elite paths.

[0146] To systematically reduce repeated access, define the "redundancy rate" as the proportion of the difference between the total number of steps and the number of unique accesses to the total number of steps, and obtain the path redundancy rate of all paths For:

[0147]

[0148] In the formula, represents the path redundancy rate of all paths , represents the total number of access steps of all paths, represents the total number of access steps of all paths, represents the set of accessed grids after deduplication of all paths .

[0149] The above formula is used to measure the proportion of repeated access of the path, and the larger the value, the more redundant it is. The redundancy rate can be directly punished in the score, or the release intensity can be reduced in the pheromone release, so as to quantitatively suppress the "circling or reciprocating" behavior and make the path as compact as possible.

[0150] The above path redundancy suppression process effectively reduces the redundant part of the path, thereby improving the simplicity and coverage efficiency of the path. Through the measurement and punishment mechanism of the redundancy rate, unnecessary repeated access is avoided, the path is more concise, the energy consumption during task execution is reduced, and time is saved in practical applications. In addition, combined with the neighborhood unvisited heuristic, the area with large coverage potential can be quickly selected, further improving the path coverage efficiency.

[0151] In step S500, based on the previous score and redundancy suppression result, enter the pheromone update stage, and use elite ranking, global reinforcement and evaporation adjustment to solidify the experience of high-quality paths into the collective memory.

[0152] Specifically, the method of releasing pheromone by weighting elite paths is as follows:

[0153] According to the path redundancy suppression result, sort the current paths, and select the top several paths as elite paths;

[0154] According to the elite path ranking result, the pheromone is released in a weighted manner, and the release intensity decreases with the size of the elite path ranking, and is normalized by the length of the elite path to prevent unnecessary long paths from gaining disproportionate advantages.

[0155] ​The release mode of the above-mentioned "coverage gain priority + length constraint" makes the high-quality path stably affect the search direction of the subsequent generations.

[0156] Further, when outside the elite path of the current generation, additional pheromone is released on the historical global optimal path to form "long-term memory", which aims to prevent the algorithm from deviating from the high-quality channel that has been verified even if the subsequent generations are disturbed randomly; the above process is represented as:

[0157] ;

[0158] In the formula, represents the historical global optimal path the pheromone increment added along each edge, represents the global reinforcement weight, represents the pheromone release constant, represents the length of the historical global optimal path.

[0159] In the above process, the pheromone increment is normalized by the path length, so that the more compact optimal path obtains stronger long-term memory reinforcement.

[0160] In this embodiment, the elite path weighting and global optimal path reinforcement mechanism ensure the long-term influence of high-quality paths. This approach ensures that historical experience can stably guide the search of subsequent paths, thereby avoiding the algorithm from falling into local optimum, enhancing the convergence and stability of path planning, and effectively reducing repeated exploration and fluctuations in path search, especially in complex or dynamic environments.

[0161] The fixed evaporation coefficient of the traditional hybrid ant colony optimization ACO cannot balance the "consolidate existing progress" and "restore diversity". The present application introduces a "feedback adjustment mechanism based on coverage progress": when the coverage of the last few generations is significantly improved, the evaporation is slowed down, and the influence of the excellent path is more lasting; when the progress stagnates, the evaporation is moderately increased, the pheromone peak concentrated too much is diluted, and the exploration ability is released again. The feedback does not require a specific function form to be unique, as long as it embodies the principle of "fast progress → retain more, slow progress → fast evaporation", which can achieve the effect of adaptive adjustment at different stages.

[0162] Specifically, the feedback adjustment mechanism based on coverage progress includes:

[0163] The coverage progress is quantified based on path coverage, the pheromone evaporation coefficient is adjusted based on the feedback mechanism of coverage progress, the progress stagnation is quantified based on coverage increment, and the pheromone release coefficient is adjusted based on coverage progress and progress stagnation.

[0164] The formula for quantifying the coverage progress based on path coverage is:

[0165] ;

[0166] ;

[0167] In the formula, Indicates the progress improvement value. and They represent the first Step and the first Step coverage, Represents candidate raster coordinates The accessed mask, This represents the candidate raster coordinates of an individual in the rasterized space during the path planning process. Represents the set of feasible units;

[0168] The formula for adjusting the pheromone evaporation coefficient based on the feedback mechanism of coverage progress is:

[0169] ;

[0170] In the formula, Indicates the pheromone evaporation coefficient. Indicates the basic volatility. The moderating factor representing the effect of control progress on the evaporation rate;

[0171] The formula for the stalled progress in quantifying incremental coverage is:

[0172] ;

[0173] In the formula, Indicators indicating stagnant progress This indicates that the coverage growth is less than the set threshold. The number of consecutive iterations;

[0174] The pheromone release coefficient is adjusted based on coverage progress and stagnation:

[0175] when At that time, the pheromone evaporation coefficient for:

[0176] ;

[0177] when At that time, the pheromone evaporation coefficient for:

[0178] ;

[0179] In the formula, and They represent and The weight.

[0180] Specifically, when the coverage of recent generations has improved significantly, the pheromone evaporation rate is slowed down, so that the effect of the best path is more lasting; while when progress stagnates, the evaporation coefficient is moderately increased to dilute the excessively concentrated pheromone peak and release more exploration capabilities.

[0181] In the process described above, this technology adjusts the pheromone evaporation rate based on coverage gain, which better consolidates paths in covered areas and effectively avoids the generation of redundant paths. For example, in the initial stage, pheromone evaporation is slower, making ants / agents more inclined to explore uncovered areas, reducing the generation of duplicate paths. As the path expands, the pheromone evaporation rate moderately increases, thereby promoting path optimization.

[0182] In this embodiment, to avoid numerical distortion and path monopoly, the process of updating pheromones also includes: applying upper and lower thresholds to the pheromone matrix obtained in each iteration of the path planning process.

[0183] Specifically, let the initial pheromone be... (Used uniform initialization); Let the pheromone release constant be... The single-generation volatility is The dimension of the elite-enhanced step size is approximately derived from the typical path length. The representative value is the path length of the historical global optimum; thus, the upper and lower threshold values ​​are expressed as follows:

[0184] ;

[0185] ;

[0186] In the formula, This represents the lower threshold of pheromones. Indicates the upper limit threshold of information. This indicates the initialization of pheromones. Indicates the initial level of pheromone release. This represents the magnitude of pheromone release per unit length. Represents the pheromone release constant, and This represents the upper limit of the global pheromone ratio.

[0187] In the above process, the lower threshold ensures that no action will be permanently eliminated due to numerical underflow, and the upper threshold prevents local paths from "monopolizing probability" due to continuous reinforcement; the dual thresholds are linked with feedback volatilization to ensure that the pheromone field evolves within a bounded and stable range, so that the fusion mechanism and elite update are both in a controllable state.

[0188] To avoid the monopoly of some edges and the near-zero probability of others, a global ratio upper limit is set to ensure that the pheromone concentration of one edge cannot be infinitely higher than that of another.

[0189] When the coverage of the generated path exceeds the task threshold (e.g., the required qualified coverage level of the scene), or when the comprehensive score does not significantly improve within a certain number of generations, the algorithm terminates and outputs the historical global optimal path, obtaining a globally optimal path with full coverage. This path has achieved a balance in the three dimensions of "coverage, length, and redundancy," and its generation process is jointly constrained by "long-term memory + ranking weighting + congestion suppression + heuristic encouragement," making it highly executable and stable.

[0190] In this embodiment, an effect verification experiment example of the above path planning method is provided.

[0191] A simulation experiment was conducted in a two-dimensional plane environment containing obstacles using Python language, and the following settings were made:

[0192] The simulation area size is a two-dimensional grid map of 10x10 units (index range 0...9).

[0193] Obstacles are set in discrete coordinate form, such as (2, 2), (5, 3), and (8, 8), corresponding to impassable grid points.

[0194] The starting point is (0, 0).

[0195] This task is full coverage planning, without a fixed endpoint. When the coverage threshold is reached, the optimal path is output early.

[0196] The overall path planning process includes:

[0197] Initialization phase: First, construct an 8-neighborhood action set and its dual direction mapping, initialize a three-dimensional pheromone tensor, and establish an visited set and taboo table structure; then load or train the PPO strategy, whose observation vector is composed of position normalization, access mask tiling, and progress ratio splicing, making the strategy output one-to-one corresponding to 8-direction actions. Set the algorithm hyperparameters as follows: number of ants 25, maximum iteration number 1000, pheromone evaporation rate ρ=0.1, release constant Q=100, ACO and PPO weights 1.5 and 2.0 respectively, neighborhood heuristic and congestion suppression enabled (congestion weight 0.3), global optimal quadratic reinforcement weight, score weight 1.0, and w_len=0.01, ranking release width 5.

[0198] Coverage search phase: In each iteration, each ant starts from the starting point and generates legal candidates (not out of bounds, not obstructed, and not in the current taboo). The number of unvisited neighbors is calculated for the candidate grid to form a heuristic gain, and crowding suppression is performed based on the frequency of occurrence of the current path to obtain a combined heuristic to encourage the expansion of new domains and suppress clustering. Then, the ACO side probability (normalized product of pheromone and heuristic) and the PPO side policy probability are calculated respectively, and multiplicative fusion and normalization are performed according to the iteration progress to form the final sampling distribution, and then the path is expanded in a sampling / greedy manner. When there are no candidates or the coverage has not increased for several steps, the individual trajectory is terminated early, and the coverage rate, geometric length and redundancy rate are summarized within the generation to calculate the comprehensive score (coverage benefit minus length cost).

[0199] Pheromone Update Phase: First, global pheromones are evaporated. Then, the top few elite paths, ranked by current rating, are released in a weighted manner, with the release intensity decreasing with ranking and normalized by path length. Pheromones are added bidirectionally along both the positive and dual directions to enhance accessibility and backtracking feasibility. Simultaneously, a secondary reinforcement is applied to the historical global best path, with its increment inversely proportional to the path length, to form a stable long-term memory framework and improve convergence robustness. The process terminates early when the coverage threshold is met or there is no significant improvement for several consecutive generations, and the optimal path and indicators are output.

[0200] Based on the above process, the final experimental results are as follows: redundancy rate 0%, coverage rate 97.94%, convergence time 30.67s, path length 107.63; among which, as Figure 2 The convergence curve shown illustrates the changes in optimal coverage and path length during the iteration process. Figure 2 In the diagram, the horizontal axis represents the number of algorithm iterations, i.e., one complete running cycle, and the vertical axis represents the coverage and length of the path in generation t; the global optimal path visualization is as follows: Figure 3 As shown, in Figure 3 In the diagram, obstacles, coverage units, and path arrows are superimposed to visually display the planning results that quickly achieve high coverage under the premise of low redundancy.

[0201] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0202] Those skilled in the art will appreciate that the embodiments described herein are presented for purposes of illustration and that the inventive principles are not limited to these particular embodiments. Other variations and modifications can be made to the embodiments without departing from the spirit and scope of the inventive principles.

Claims

1. A near-end policy enhanced ant colony optimization path coverage method, characterized in that, The method comprises the following steps: S100, modeling a working environment as a raster space, initializing a visited set and a taboo table in a path planning process, and constructing an observation vector of an individual in the working environment when path planning is performed by using a proximal policy optimization (PPO); S200, performing iterative path planning in the raster space based on a combination of a hybrid ant colony optimization (ACO) and the PPO; In each iteration, a number of unvisited neighborhoods in a current candidate path generation process is calculated to form a heuristic multiplier, and a frequency of the current candidate path is calculated to form a congestion correction term, and the heuristic multiplier and the congestion correction term are used as a combined heuristic; S300, calculating, according to the combined heuristic, a probability of each candidate action when a candidate path is generated in each iteration when global path planning is performed by using the ACO, and calculating, according to the observation vector of the individual, the probability of each candidate action when the candidate path is generated in each iteration when local path planning is performed by using the PPO; The probabilities of the candidate actions are adaptively fused according to an iteration progress to generate an action decision probability, and then path expansion is performed in the iteration process in an action sampling or greedy manner to generate a global path, and the taboo table and the visited set are synchronously updated; S400, performing path redundancy suppression based on a path coverage rate and a path redundancy rate of the global path; S500, releasing pheromone in the path planning process by an elite path weighting and a global optimal path reinforcement mechanism based on a path redundancy suppression result, and introducing a feedback regulation mechanism based on coverage progress to adaptively update a volatility rate of the pheromone until a coverage rate of a generated path meets a set task threshold to obtain a global optimal path with full coverage.

2. The near-end policy enhanced ant colony optimization path cover method according to claim 1, characterized in that, In the step S100, when the agent is in the position , the observation vector of the agent is: ​ ； ； ； ； ； wherein, represents the current position normalized vector, represents the visited binary map tiling vector, represents the progress proportion vector, represents the step number, represents the visited mask, represents the matrix vectorization operator, represents the progress fusion coefficient controlling the coverage and step number weight, represents the coverage of the step, represents the relative step progress, represents the total dimension of the observation vector, , represents the length and width of the rasterized space, represents the position coordinates of the individual in the rasterized space at the step, represents the discrete state set corresponding to the rasterized space.

3. The near-end policy enhanced ant colony optimization path cover method according to claim 1, characterized in that, In the step S200, the heuristic multiplier is represented as: ； ； wherein, represents a heuristic value of a candidate action, represents a heuristic weight factor, represents a candidate grid point of an individual in a path planning process the number of unvisited feasible neighbors in its 8-neighborhood, represents a grid point an indicator function whether a feasible cell set is visited, represents a grid point an indicator function whether a grid point is visited, represents a certain neighbor grid point of a candidate grid point represents a set of 8-neighbor grid points of a candidate grid point .​ The congestion correction term is represented as: ； ； In the formula, represents the number of paths containing the grid point j in the step iteration, represents the number of paths containing the grid point j in the step iteration, represents the congestion suppression weight, represents the number of paths containing the grid point j in the step iteration, represents the path set generated by all individuals at the step iteration, represents the grid point set after deduplication of the path grid point set, represents the maximum value function, represents the generated path index.

4. The near-end policy enhanced ant colony optimization path cover method according to claim 1, wherein, In the step S300, the probability of each candidate action when the candidate path is generated when the global path planning is performed by using the ACO is represented as: ； ； wherein denotes the probability of moving from grid point to a candidate next grid point at step denotes the pheromone intensity on the edge from grid point to candidate next grid point at step denotes the pheromone intensity on the edge from grid point to candidate next grid point at step denotes the combination heuristic for candidate actions, the superscripts and denote the pheromone heuristic factor and the desirability heuristic factor, respectively, denotes another candidate next grid point different from candidate next grid point denotes the set of legal candidate grid points for grid point at step denotes the set of legal candidate grid points for grid point at step In the step S300, the probability of each candidate action when the candidate path is generated when the local path planning is performed by using the PPO is represented as: ； In the formula, denotes the action probability of the original output of the policy network when the proximal policy optimization PPO is adopted, the probability of the agent moving from the grid point to the candidate next grid point , denotes the action probability of the original output of the policy network when the proximal policy optimization PPO is adopted, denotes the candidate mask of the candidate next grid point , denotes the action probability of the original output of the policy network when the proximal policy optimization PPO is adopted, at the first step the agent moving from the grid point to the candidate next grid point , denotes the candidate mask of the candidate next grid point Adaptively fuse the probability of candidate actions with the progress of iterations to generate action decision probability is represented as: ； In the formula, denotes a fusion factor that varies with the progress of iteration.

5. The near-end policy enhanced ant colony optimization path cover method according to claim 1, wherein, The path coverage rate of the whole path in the step S400 is: ； wherein denotes the coverage of all paths , denotes the weight of , denotes the length of all paths , denotes the weight of ; Path redundancy rate of all paths For: ； In the formula, the path redundancy rate of all paths, the path redundancy rate of all paths, the total number of access steps of all paths, the path redundancy rate of all paths, the set of visited grids after deduplication.

6. The near-end policy enhanced ant colony optimization path cover method according to claim 1, wherein, In the step S500, a method for releasing the pheromone by the elite path weighting is: The current paths are sorted according to a path redundancy suppression result, and the first several paths are selected as elite paths; The pheromone is released by weighting according to an elite path sorting result, and a release strength decreases with an elite path sorting size, and is normalized by an elite path length.

7. The near-end policy enhanced ant colony optimization path cover method according to claim 1, wherein, In the step S500, the historical global optimal path is additionally released pheromone, which is represented as: ； wherein denotes the historical global optimal path denotes the pheromone increment added along each edge, denotes the global reinforcement weight, denotes the pheromone release constant, denotes the length of the historical global optimal path.

8. The near-end policy enhanced ant colony optimization path cover method according to claim 1, wherein, In the step S500, the feedback regulation mechanism based on the coverage progress comprises: The coverage progress is quantified based on a path coverage rate, a pheromone volatility coefficient is adjusted based on a feedback mechanism based on the coverage progress, progress stagnation is quantified based on a coverage rate increment, and a pheromone release coefficient is adjusted based on the coverage progress and the progress stagnation.

9. The near-end policy enhanced ant colony optimization path cover method according to claim 8, characterized in that, A formula for quantifying the coverage progress based on the path coverage rate is: ； ； wherein, denotes the progress boost value, and denote the coverage of the first and second steps, respectively, denote the coverage of the first and second steps, respectively, denotes the visited mask of candidate grid coordinates denotes the candidate grid coordinates of the individual in the rasterized space during the path planning process, denotes the set of feasible cells; A formula for adjusting the pheromone volatility coefficient based on the feedback mechanism based on the coverage progress is: ； wherein represents the pheromone volatility coefficient, represents the base volatility rate, represents the adjustment coefficient of the influence of the control progress on the volatility rate; A formula for quantifying the progress stagnation based on the coverage rate increment is: ； wherein represents a stagnation of progress indicator, represents a coverage growth is less than a set threshold of consecutive iteration steps; The pheromone release coefficient is adjusted based on the coverage progress and the progress stagnation. When the pheromone volatility coefficient is: ； When the pheromone volatility coefficient is: ； wherein and respectively represent and the weight of 10. The near-end policy enhanced ant colony optimization path cover method according to claim 8, wherein, In the process of updating pheromone, also includes: the path planning process in each round of iteration update the pheromone matrix is imposed on the upper and lower limit of double threshold, which is expressed as: ； ； wherein represents the pheromone lower threshold, represents the pheromone upper threshold, represents the initialization pheromone, represents the initialization level of pheromone release, represents the unit length release magnitude of pheromone, represents the pheromone release constant, and represents the global ratio upper limit of pheromone.

Images