Path planning method, device and electronic equipment

Abstract

Embodiments of the present application provide a path planning method and device and electronic equipment, the method comprises: constructing at least two populations;For any population, obtain the evaluation result of the current evolution guiding item corresponding to the population;Based on the evaluation results of various populations, the evaluation result is not lower than the candidate path of other candidate paths, and the target path of the population is determined, and the target path is added to the candidate set;Based on the evaluation results of each candidate path in the population to be contracted, the constraint violation degree not lower than other candidate paths is set as the constraint boundary of the population to be contracted;For the population, based on the evaluation results of the population and the constraint boundary, the candidate path in the population is evolved to obtain a new population;For each new population, re-execute the operation of obtaining the evaluation result of the current evolution guiding item corresponding to the population, until the stop condition is reached, the target path contained in the candidate set is determined as the final path. Improve the path planning effect.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a path planning method, apparatus, and electronic device. Background Technology

[0002] With the development of robotics technology, constrained multi-objective evolutionary algorithms are commonly used in the field of multi-agent robot path planning. Related technologies often employ a single population to simultaneously handle constraint satisfaction and objective optimization.

[0003] However, this approach forces the population to simultaneously consider feasibility and optimality during the evolution process, leading to problems such as insufficient search of feasible regions, an excessively high proportion of infeasible solutions, and slow convergence speed, resulting in poor path planning performance. Summary of the Invention

[0004] This invention provides a path planning method, apparatus, and electronic device to solve the problem of poor path planning performance in related technologies.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is implemented as follows: In a first aspect, the present invention provides a path planning method, the method comprising: Construct at least two populations; each population contains a preset number of candidate paths for the target robot; For any of the populations, obtain the current evaluation result of the evolutionary guidance term corresponding to the population; different populations have different evolutionary guidance terms, and the evolutionary guidance term includes at least one of constraint violation degree and fitness. Based on the evaluation results of each population, the candidate paths whose evaluation results are not lower than other candidate paths are determined as the target paths of the population, and the target paths are added to the candidate set. Based on the evaluation results of each candidate path in the population to be shrunk, a constraint violation degree no lower than that of other candidate paths is set as the constraint boundary of the population to be shrunk; the population to be shrunk is a population whose evolutionary guidance term includes the constraint violation degree; wherein, the constraint boundary of the first type of population is larger than the constraint boundary of the second type of population, the first type of population is a population including at least two evolutionary guidance terms, and the second type of population is a population including one evolutionary guidance term. For the population, based on the evaluation results and constraint boundaries of the population, the candidate paths in the population are subjected to evolutionary processing to obtain a new population; For each new population, the operation of obtaining the current evaluation result of the evolutionary guidance item corresponding to the population is re-executed until the stopping condition is met, and then each target path contained in the candidate set is determined as the final path.

[0006] In a second aspect, the present invention provides a path planning device, the device comprising: Construct at least two populations; each population contains a preset number of candidate paths for the target robot; For any of the populations, obtain the current evaluation result of the evolutionary guidance term corresponding to the population; different populations have different evolutionary guidance terms, and the evolutionary guidance term includes at least one of constraint violation degree and fitness. Based on the evaluation results of each population, the candidate paths whose evaluation results are not lower than other candidate paths are determined as the target paths of the population, and the target paths are added to the candidate set. Based on the evaluation results of each candidate path in the population to be shrunk, a constraint violation degree no lower than that of other candidate paths is set as the constraint boundary of the population to be shrunk; the population to be shrunk is a population whose evolutionary guidance term includes the constraint violation degree; wherein, the constraint boundary of the first type of population is larger than the constraint boundary of the second type of population, the first type of population is a population including at least two evolutionary guidance terms, and the second type of population is a population including one evolutionary guidance term. For the population, based on the evaluation results and constraint boundaries of the population, the candidate paths in the population are subjected to evolutionary processing to obtain a new population; For each new population, the operation of obtaining the current evaluation result of the evolutionary guidance item corresponding to the population is re-executed until the stopping condition is met, and then each target path contained in the candidate set is determined as the final path.

[0007] Thirdly, the present invention provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect above.

[0008] Fourthly, the present invention provides a readable storage medium that, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method described in the first aspect.

[0009] The path planning method provided in this embodiment of the invention can construct at least two populations; each population contains a preset number of candidate paths for the target robot; for any population, the current evaluation result of the evolutionary guidance term corresponding to the population is obtained; different populations correspond to different evolutionary guidance terms, and the evolutionary guidance term includes at least one of constraint violation degree and fitness; based on the evaluation results of each population, candidate paths with evaluation results not lower than other candidate paths are determined as the target paths of the population, and the target paths are added to the candidate set; based on the evaluation results of each candidate path in the population to be shrunk, the constraint violation degree not lower than other candidate paths is set as the target path. The constraint boundary of the population to be shrunk; the population to be shrunk is a population whose evolutionary guidance terms include the constraint violation degree; wherein, the constraint boundary of the first type of population is larger than the constraint boundary of the second type of population, the first type of population includes at least two evolutionary guidance terms, and the second type of population includes one evolutionary guidance term; for the population, based on the evaluation results of the population and the constraint boundary, the candidate paths in the population are subjected to evolutionary processing to obtain a new population; for each new population, the operation of obtaining the current evaluation results of the evolutionary guidance terms corresponding to the population is re-executed until the stopping condition is met, and then each target path contained in the candidate set is determined as the final path. In this embodiment of the invention, by setting multiple populations and using different evolutionary guidance terms for different populations, the problem of not being able to take into account the individual in the evolution process caused by using only a single population for optimization in related technologies is avoided. Furthermore, in this embodiment of the invention, by setting a first type of population and a second type of population, the population is divided into a unicellular population and a multicellular population. The unicellular population is guided by only one evolutionary guide, while the multicellular population considers multiple evolutionary guides simultaneously. Furthermore, by setting the constraint boundary of the first type of population to be larger than that of the second type of population, the constraint boundaries of the two populations remain highly correlated. Through the division of labor and cooperation between the two types of populations, the unicellular population can quickly screen feasible solutions, while the multicellular population can further optimize the objective based on feasible solutions. This addresses the inability of a single population to smooth the contradiction between constraints and objectives, improves the efficiency of feasible region search and the quality of solutions, and enhances the effectiveness of path planning. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart of the steps of a path planning method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a constraint boundary provided in an embodiment of the present invention; Figure 3 This is the constraint boundary constraint contraction process provided in the embodiments of the present invention; Figure 4 This is a schematic diagram of a relational function image of state information provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a preset state table provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of a path planning process provided by an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a path planning device provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, the first object can be one or more. Furthermore, the term "and / or" in the specification and claims is used to describe the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. In embodiments of this invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0014] Figure 1 This is a flowchart of the steps of a path planning method provided in an embodiment of the present invention, as follows: Figure 1 As shown, the method may include the following steps: Step 101: Construct at least two populations; each population contains a preset number of candidate paths for the target robot.

[0015] Step 102: For any of the populations, obtain the current evaluation result of the evolutionary guidance item corresponding to the population; different populations correspond to different evolutionary guidance items, and the evolutionary guidance item includes at least one of constraint violation degree and fitness.

[0016] Step 103: Based on the evaluation results of each population, determine the candidate paths whose evaluation results are not lower than other candidate paths as the target paths of the population, and add the target paths to the candidate set.

[0017] Step 104: Based on the evaluation results of each candidate path in the population to be shrunk, set the constraint violation degree, which is not lower than that of other candidate paths, as the constraint boundary of the population to be shrunk; the population to be shrunk is a population whose evolutionary guidance term includes the constraint violation degree; wherein, the constraint boundary of the first type of population is greater than the constraint boundary of the second type of population, the first type of population is a population that includes at least two evolutionary guidance terms, and the second type of population is a population that includes one evolutionary guidance term.

[0018] Step 105: For the population, based on the evaluation results and constraint boundaries of the population, perform evolutionary processing on the candidate paths in the population to obtain a new population.

[0019] Step 106: For each new population, re-execute the operation of obtaining the current evaluation result of the evolutionary guidance item corresponding to the population until the stopping condition is met, and then determine each target path contained in the candidate set as the final path.

[0020] Regarding steps 101-106 above, this embodiment of the invention can be applied to any device with computing power. The target robot can be a robot requiring path planning, and there can be one or more. Furthermore, the target robot can be a robot applicable to any field, such as an industrial robot, a home robot, etc., and this embodiment of the invention does not impose any limitations on this.

[0021] The preset quantity can be set according to actual needs. The preset quantity can be represented by N. For example, N can be 50, 100, 150, etc., and can be set according to actual needs. This embodiment of the invention does not limit this.

[0022] The candidate paths mentioned above can be solutions to a path planning problem for the target robot. Specifically, multiple populations, each containing N candidate paths, can be randomly generated. The evolutionary guidance term can be the evolutionary goal of the population. Specifically, different populations can have different evolutionary guidance terms, and each population's evolutionary guidance term can include at least one of constraint violation degree and fitness. For example, this embodiment of the invention can construct two populations. The evolutionary guidance term for one population can include constraint violation degree, that is, this population only uses constraint violation degree as its evolutionary guide. Correspondingly, the evolutionary guidance term for the other population can include both constraint violation degree and fitness, that is, this population considers both constraint violation degree and fitness during the evolutionary process.

[0023] The constraint violation degree described above represents an indicator of whether the solution is feasible when solving the corresponding problem. Only when the constraint violation degree is 0 does it mean that the solution satisfies the constraints of the current subproblem. The fitness degree described above can be a pre-set value of an objective function, which can be path length, path time, path safety, path smoothness, collaboration time, etc., and can be set according to actual needs. This embodiment of the invention does not impose any restrictions on this.

[0024] It should be noted that robot path planning refers to multi-robot path planning from the starting point to the target point. This problem is a complex multi-objective, multi-constraint optimization problem, which traditional methods often struggle to solve effectively. The embodiments of this invention can efficiently handle equality and inequality constraints. In the algorithm, an auxiliary problem is constructed to handle the constraints, dividing the constraints of the original problem into multiple actions and states, enabling the algorithm to traverse infeasible regions in the target space.

[0025] It should be noted that different cell types of organisms typically exist in an ecosystem. Unicellular organisms are omnipotent individuals, encompassing all functions and completing tasks independently of other individuals. Multicellular organisms, on the other hand, have a high degree of specialization, performing different tasks in an orderly manner at different stages. Based on the above description, embodiments of the present invention can divide populations into unicellular and multicellular populations for separate path planning calculations. In this case, the first type of population can be called a multicellular population, and the second type of population can be called a unicellular population. That is, the first type of population is one that simultaneously considers at least two evolutionary guiding factors, while the second type of population is one that only uses one evolutionary guiding factor to achieve its goal.

[0026] Specifically, the degree of constraint violation mentioned above can be calculated using the following formula: .

[0027] in, This is the tolerance used to relax the equality constraints, which can be set according to actual needs. This embodiment of the invention does not impose any restrictions on this. Wherein, g i This represents a series of constraints that require g(x) <= 0 and their related constraints. For example, assuming a speed limit of 5 m / s, it can be represented as the i-th problem: g(x) = v(x) - 5 <= 0. Where h... i This represents a set of constraints that require h(x) = 0 and their related constraints. Specifically, if CV(X) = 0, then solution X is feasible; otherwise, X is infeasible.

[0028] It should be noted that directly setting the absolute value to 0 may cause calculation problems, therefore a tolerance is set. .

[0029] Alternatively, an obstacle avoidance formula may be used: .

[0030] Where K represents the total number of robots, D represents the total number of waypoints for each robot, and there are T threat zones within the range. This is the robot's safe radius. Let p be the i-th path point of the k-th robot. ik To the center c of the j-th threat zone j The distance d. ijk <r j This indicates that the robot is within the threat zone, and max(r) j d ijk If d > 0. ijk ≥r j If the robot is outside the threat zone, then max(r) j d ijk ,0) = 0. All path points must satisfy d ijk ≥r j .

[0031] Furthermore, the degree of constraint violation can be determined by... Inequality constraints and It consists of several equality constraints. For simplicity, in embodiments of the present invention, the original problem, the single-cell auxiliary problem, and the multi-cell auxiliary problem can be respectively denoted as... , and .

[0032] Specifically, the original robot path planning problem contains numerous equality and inequality constraints, making direct optimization very complex. While ignoring constraints is simpler than the original problem, its weak connection to the original problem makes it ineffective in guiding population evolution. The original problem is addressed collaboratively by solving a series of auxiliary problems derived from the original problem. The original problem refers to robot path planning considering all constraints. After constraint processing (such as in multi-cell population algorithms), a series of problems where constraints gradually tighten with the boundary can be considered as auxiliary / sub-problems. The auxiliary problems are not derived by solving the original problem directly; solving the original problem (considering all constraints) is very complex and computationally inefficient. Solving the auxiliary problems approximates the original problem, accelerating computation. As the constraint boundary shrinks, the difficulty of solving constraint-focused problems at each stage gradually increases. During the evolutionary process, each generation constructs multi-cell auxiliary problems to effectively solve the single-cell auxiliary problems of the current stage.

[0033] Furthermore, in this embodiment of the invention, based on the evaluation results of each candidate path in the population to be shrunk, a constraint violation degree no lower than that of other candidate paths can be set as the constraint boundary of the population to be shrunk. The population to be shrunk can be a population whose evolutionary guidance term includes constraint violation degree.

[0034] For example, in the embodiments of the present invention, all single-cell populations can be initialized during the population initialization stage. and multicellular populations The initial maximum constraint violation, that is and As a subproblem and The initial boundary is defined as follows: The initial violation degree is the constraint violation degree obtained after calculating the original problem during initialization. Generally, the initial violation degree is a relatively large value because all constraints are considered, but the calculated solution does not meet the conditions. Two populations can be calculated under a certain constraint violation degree to ensure computational efficiency. As the number of iterations increases, the constraint violation degree considered as the boundary can be gradually tightened, and the calculation results can gradually approach the real problem. When all solutions in the population meet the constraint requirements, this embodiment of the invention can proceed to the next stage, and the subproblem can be adapted to the new constraint boundary. .

[0035] Optionally, Figure 2 This is a schematic diagram of a constraint boundary provided in an embodiment of the present invention, such as... Figure 2 As shown, to ensure a high correlation between unicellular and multicellular populations during evolution, it is necessary to maintain the relationship between unconstrained and constrained objective functions within the population. Therefore, the constraint boundary of the multicellular population at each stage... Slightly larger than the constraint boundary of a single-cell population The update frequency is also lower. Figure 2The middle part constructs highly correlated constraints for each evolutionary stage. and Examples are shown. The subgraphs represent the search spaces for different problems after relaxing constraints. The gray area represents the feasible region, the thick black line below represents the unconstrained Pareto front, and the dashed line represents the constrained Pareto front.

[0036] The aforementioned stopping condition can be that the constraint boundary is lower than a preset threshold, or that the number of iterations reaches a preset threshold. Of course, other conditions can also be set, which can be set according to actual needs. This embodiment of the invention does not limit this.

[0037] It should be noted that current multi-agent robot path planning generally employs constrained multi-objective evolutionary algorithms for solving problems. Related technologies primarily utilize single-population or simple multi-population optimization methods based on the second-generation non-dominated sorting genetic algorithm (NSGA-II), linear sedation strategies with fixed constraint boundaries, and optimization frameworks with fixed evolutionary operator parameters. While these methods can complete basic path planning tasks under conventional constraints, they exhibit significant limitations when facing complex scenarios and multi-robot dynamic cooperative path planning problems with coupled constraints. For example, in agricultural inspection or emergency rescue scenarios involving dynamic obstacles, dynamic collision avoidance, path length and balance constraints, and multi-robot temporal cooperative constraints, traditional algorithms use a single population to simultaneously handle constraint satisfaction and objective optimization. This forces the population to simultaneously consider feasibility and optimality during evolution, leading to problems such as insufficient feasible region search, an excessively high proportion of infeasible solutions, and slow convergence speed. Meanwhile, related technologies generally employ fixed-parameter operators, failing to adaptively switch between global search and local exploration modes based on the current evolutionary state of the population, iteration progress, and solution distribution characteristics. This makes it difficult for the algorithm to fully utilize the search space in the early stages of evolution, while it is prone to premature convergence due to local optima in the later stages. Furthermore, most existing constraint handling methods use fixed-rate boundary decay strategies. The contraction process of the constraint boundary is independent of the actual evolutionary state of the population, the proportion of feasible solutions, and the degree of constraint violation. For example, directly using strict constraint boundaries in the early stages of iteration significantly compresses the effective search range, while maintaining loose boundaries in the later stages prevents the algorithm from quickly converging to the true feasible region. Ultimately, this results in path planning defects such as tortuous paths, excessive length, and unmet constraints. More importantly, these technologies almost never effectively integrate reinforcement learning with evolutionary algorithms, failing to achieve autonomous decision-making and dynamic adjustment of evolutionary strategies. Faced with complex and ever-changing path planning environments, they struggle to autonomously select the optimal search behavior, further reducing the algorithm's optimization accuracy and stability under multi-constraint and multi-objective conditions.

[0038] The path planning method provided in this embodiment of the invention can construct at least two populations; each population contains a preset number of candidate paths for the target robot; for any population, the current evaluation result of the evolutionary guidance term corresponding to the population is obtained; different populations correspond to different evolutionary guidance terms, and the evolutionary guidance term includes at least one of constraint violation degree and fitness; based on the evaluation results of each population, candidate paths with evaluation results not lower than other candidate paths are determined as the target paths of the population, and the target paths are added to the candidate set; based on the evaluation results of each candidate path in the population to be shrunk, the constraint violation degree not lower than other candidate paths is set as the target path. The constraint boundary of the population to be shrunk; the population to be shrunk is a population whose evolutionary guidance terms include the constraint violation degree; wherein, the constraint boundary of the first type of population is larger than the constraint boundary of the second type of population, the first type of population includes at least two evolutionary guidance terms, and the second type of population includes one evolutionary guidance term; for the population, based on the evaluation results of the population and the constraint boundary, the candidate paths in the population are subjected to evolutionary processing to obtain a new population; for each new population, the operation of obtaining the current evaluation results of the evolutionary guidance terms corresponding to the population is re-executed until the stopping condition is met, and then each target path contained in the candidate set is determined as the final path. In this embodiment of the invention, by setting multiple populations and using different evolutionary guidance terms for different populations, the problem of not being able to take into account the individual in the evolution process caused by using only a single population for optimization in related technologies is avoided. Furthermore, in this embodiment of the invention, by setting a first type of population and a second type of population, the population is divided into a unicellular population and a multicellular population. The unicellular population is guided by only one evolutionary guide, while the multicellular population considers multiple evolutionary guides simultaneously. Furthermore, by setting the constraint boundary of the first type of population to be larger than that of the second type of population, the constraint boundaries of the two populations remain highly correlated. Through the division of labor and cooperation between the two types of populations, the unicellular population can quickly screen feasible solutions, while the multicellular population can further optimize the objective based on feasible solutions. This addresses the inability of a single population to smooth the contradiction between constraints and objectives, improves the efficiency of feasible region search and the quality of solutions, and enhances the effectiveness of path planning.

[0039] Optionally, the operation of performing evolutionary processing on the candidate paths in the population based on the evaluation results and constraint boundaries to obtain a new population, as described above, may specifically include: S21. Based on the evaluation results of the population and the constraint boundary, select the parent population from the population using the binary tournament selection method.

[0040] S22. Based on the parent population, perform evolutionary processing to generate the offspring population.

[0041] S23. Merge the population with the offspring population to obtain the population to be selected.

[0042] S24. Based on the constraint boundary, perform environmental selection on the population to be selected to obtain a new population.

[0043] Specifically, the aforementioned binary tournament selection method refers to randomly selecting two individuals (candidate paths) to compare their merits and keep the best one as the parent generation.

[0044] Specifically, different types of populations can undergo different evolutionary operations. For example, for the first type of population, Q-learning can be used to evolve the parent population to generate the offspring population. Correspondingly, for the second type of population, a genetic algorithm can be used to evolve the parent population to generate the offspring population.

[0045] For example, embodiments of the present invention can employ a binary tournament selection method to select from... and Select two parent populations and For population By simulating binary cross The generation of offspring population with the polynomial mutation operator For population The Q-table is used to learn operations to generate offspring. Then , Each population is merged with its corresponding offspring population to obtain the population H to be selected.

[0046] The aforementioned environment selection can be based on prioritizing feasible solutions (i.e., solutions that do not exceed the constraint boundaries) within the constraint boundary, and then selecting N individuals according to the dominance and density relationship between solutions. First, solutions that satisfy the constraints are selected; if insufficient, N individuals are selected based on dominance and non-repetition.

[0047] In this embodiment of the invention, the population can be automatically optimized through the above method, thereby improving the quality of candidate paths.

[0048] Optionally, before the operation of selecting the parent population from the population using a binary tournament selection method based on the evaluation results and constraint boundaries of the population, the embodiments of the present invention may further include: The constraint boundary of the population to be shrunk is shrunk in the following manner: ; Among them, the Let t represent the initial boundary of any candidate path with respect to the j-th constraint, and let t represent the iteration number. The preset contraction rate is defined as j = 1, 2, ..., m, where m is the number of constraints on the target robot, and Tmax is a preset iteration threshold, which can be set based on actual iteration requirements. For example, the iteration threshold can be 1000. Optionally, in the case of unconstrained multi-target scenarios, the iteration threshold can also be set to 200; this embodiment of the invention does not impose any limitations on this. The preset contraction rate can be set according to actual exploration needs, for example, it can be set to 25, 50, 100, 200, etc. Furthermore, to ensure sufficient exploration in the early stage and sufficient contraction in the later stage during boundary contraction, resulting in a relatively gentle early and late stage with a less steep rate decrease in the middle stage, this embodiment of the invention, through experimental verification, shows that the preset contraction rate can be set to 50.

[0049] Specifically, during the population initialization phase, all single-cell populations are... and multicellular populations Initial maximum constraint violation and As a subproblem and The initial boundary is determined. When all solutions in the population satisfy the constraints, the optimization algorithm moves to the next stage, and the subproblems are adapted to the new constraint boundaries. .

[0050] Specifically, embodiments of the present invention can be implemented using the above formula generated based on the activation function (Sigmoid). It should be noted that the basic expression of the activation function is as follows: .

[0051] Since the domain of the standard Sigmoid function is the real number field R, and its range is the open interval (0,1), its function value only approaches 0 or 1 infinitely as the independent variable approaches positive or negative infinity. Therefore, it cannot directly adapt to the closed interval value requirement of boundary contraction. Thus, embodiments of the present invention can perform shifting and scaling transformations on the basic Sigmoid function to ensure that its output range strictly matches the preset upper and lower limits. For example, Figure 3 This is the constraint boundary constraint shrinkage process provided in the embodiments of the present invention. It can be seen that as the number of iterations t increases, from Gradually decreasing to near 0 indicates that the maximum constraint violation gradually decreases and gets closer to the constraints of the original problem. It should be noted that in this embodiment of the invention, the constraint boundary can be gradually tightened, and therefore continuous. However, the maximum constraint violation actually consists of p+q constraints, which are discrete. This embodiment of the invention can use continuously tightening boundary conditions to represent the stage from the initial stage of ignoring constraints to the final stage of satisfying the constraints of the original problem. Simultaneously, as t approaches 0, Approaching When t approaches Tmax, Approaching 0, there is a very small tolerance between the two. Regarding the equality constraint, Reducing the value to 0 may lead to inaccurate final population selection. If the value is directly equal to 0, or too small, matrix singularity may occur. Therefore, embodiments of the present invention can also employ [a specific method] in the termination iteration. With a small tolerance, solutions with CV values ​​below the threshold are considered feasible.

[0052] Furthermore, in this embodiment of the invention, single-cell populations tend to choose solutions with lower constraint violation rates, thus favoring feasible solutions for robot swarms. Multicellular populations, however, not only select solutions with lower constraints but also solutions with better objective values, aiming to obtain the optimal solution for the robot swarm. (Regarding the population...) If mixed population The number of feasible solutions exceeds Then, the Pareto evolutionary algorithm's environment selection mechanism is used to select from all feasible solutions. One solution; otherwise, based on the degree of constraint violation... Select One solution.

[0053] In this embodiment of the invention, a constraint boundary contraction strategy enhances the algorithm's early-stage exploration and later-stage development capabilities. In the early stages of evolution, the constraint boundary decreases slowly, allowing the algorithm to traverse the entire search space with fewer constraints. In the middle stages of evolution, the constraint boundary ζ contracts more rapidly, shifting the search focus from objective optimization to constraint satisfaction. In the later stages of evolution, the constraint boundary decreases to near zero, enabling the algorithm to better develop the optimal feasible region. The strong match between the constraint boundary and the evolutionary stage improves constraint processing efficiency, making the algorithm more stable and smoothly approaching the optimal feasible solution in complex multi-path planning. This avoids the problems in related technologies where the constraint boundary decays in a fixed manner, the decay rate is independent of the population's evolutionary state, and excessively tight constraints in the early stages lead to a limited search space, while excessively loose constraints in the later stages lead to slow convergence.

[0054] Optionally, embodiments of the present invention may also employ strategies such as linear reduction or piecewise reduction to shrink the constraint boundary. These strategies can be set according to actual needs, and embodiments of the present invention do not impose any restrictions on them.

[0055] Optionally, in this embodiment of the invention, for a given constrained multi-objective optimization problem, this embodiment of the invention can refactor it into the following constrained multi-objective optimization problem: ,(1); (2); (3); , (4).

[0056] in, ,in, The maximum number of iterations, This represents the relaxation boundary of the j-th constraint in the t-th iteration, i.e. The constraint range. Here, F(x) represents a set of objective functions that the algorithm needs to solve, f represents a single distinct objective function, such as the total path length, threat level of the threat area, etc. x represents a series of design variables that need to be satisfied, l i with u i Used to represent the upper and lower bounds of x. g and h represent the constraints that need to be satisfied. g represents a series of constraints where g(x) <= 0 and their related constraints. h represents a series of constraints where h(x) = 0 and their related constraints.

[0057] Among them, the above formulas (1) to (4) are an overview of the overall objective function and constraint function of the optimization algorithm. Here, F(x) represents a set of objective functions that the algorithm needs to solve, and f represents a single, different objective function, such as the total path length or the threat level of a threat area. X represents a series of design variables that need to be satisfied, x... i Let l represent the i-th design variable. i with u i They represent x respectively i The upper and lower bounds define x. i The range of values ​​for t is 0, 1, ..., Tmax, where t = 0, 1, ..., Tmax represents the number of iterations, and Tmax represents the maximum number of iterations. Optionally, after adding the target path to the candidate set as described above, the embodiments of the present invention may further include: S31. Update the preset state table; the preset state table includes value estimates for different state action pairs.

[0058] If the population is the first type of population, the above-mentioned operation of performing evolutionary processing based on the parent population to generate the offspring population may specifically include, in this embodiment of the invention: S32. Based on the preset state table, perform evolutionary learning on the parent population to generate the offspring population.

[0059] Specifically, in this embodiment of the invention, for the first type of population or multicellular population, an evolutionary process can be performed using a preset state table. The preset state table can be a Q-table, and correspondingly, the evolutionary process can be a Q-learning method.

[0060] Specifically, Q-learning is a reinforcement learning algorithm proposed by Watkins. It uses a Q-table to guide the parameters influencing navigation points, and further uses these parameters to guide a control strategy for optimizing navigation point position parameters. This allows the algorithm to adaptively adjust selection parameters during the search process, improving overall optimization performance. In this invention, embodiments can implement a multi-objective optimization algorithm based on Q-learning through state definition, action definition, reward definition, and Q-table updates. In these embodiments, Q-learning can control multi-cellular populations for optimization.

[0061] Optionally, the above-described operation of updating the preset state table may specifically include, in this embodiment of the invention: S41. For each navigation point, determine the reward value of the navigation point based on the difference in fitness between the navigation point after the path update and before the update.

[0062] S42. Based on the value estimate of the navigation point under the current state action pair, the reward value, the preset learning rate and discount factor, and the maximum value estimate of the navigation point under the target state action pair, determine the current value estimate of the navigation point under the target state action pair.

[0063] Optionally, the state information in the above state-action pair is set based on the following method: S51. For any navigation point, calculate the state information of the navigation point based on the Euclidean distance between the navigation point and the global optimal navigation point, the Euclidean distance between the navigation point and the individual historical optimal navigation point, and the obstacle factor.

[0064] S52. Normalize the state information to obtain discrete state information, and set the state information in the preset state table as the discrete state information.

[0065] Specifically, in Q-learning, state information can be used to characterize environmental problems or environmental information. In this embodiment of the invention, the Euclidean distance between the i-th location and the t-th generation of the navigation point and the globally optimal location can be used as the state information of the current navigation point. This embodiment of the invention can calculate the state information of the navigation point based on the following formula: .

[0066] Among them, the above Indicates status information. This indicates the location of the navigation point relative to the globally optimal location (globally optimal navigation point). The Euclidean distance between them. This indicates the current navigation point position relative to the individual's historical best position (the individual's historical best navigation point). The distance between them.

[0067] .

[0068] .

[0069] in, This represents the position of the navigation point at the i-th position and the t-th generation.

[0070] The barrier multiplier (barrier factor) can be represented as: .

[0071] Wherein, eps can be an extremely small number, such as 10^(-8). It should be noted that since Euclidean distance is often non-negative, the absolute value condition of the variables in the logarithmic function can be disregarded in this embodiment of the invention.

[0072] For example, Figure 4 This is a schematic diagram of a relational function image of state information provided in an embodiment of the present invention, such as... Figure 4 As shown, the graph illustrates the relationship between state information, individual historical distance, and global optimal distance. Figure 4 As shown, when the current waypoint differs significantly from the individual or globally optimal waypoint, the state information changes noticeably. If the distance fluctuates only within a limited range, the state information changes only slightly. This embodiment of the invention can distinguish the state of navigation points in this way, allowing for better differentiation of different waypoint situations.

[0073] Furthermore, embodiments of the present invention can discretize the state of the Q-table, and... Normalization yields This is used to identify the four different state information that are normalized and then averaged and discretely divided:

[0074] Specifically, the normalization process in the embodiments of the present invention can satisfy... The embodiments of the present invention can be defined =0.75, =0.5, =0.25, define the state set S={ , , , } arrive These represent four states of relationship with the optimal solution, ranging from approaching to becoming estranged. This discretization method simplifies the characteristics of complex environments by comprehensively considering the distance between the navigation point and the global optimal solution, as well as classifying environmental obstacles into different levels, enabling the algorithm to adaptively adjust parameters based on the relative status of the navigation point.

[0075] In this embodiment of the invention, by determining the state information of the Q-table in the above manner, the state can be defined according to the distance difference between the navigation point and the individual optimal and global optimal, the performance differences under different states can be distinguished, and the effect of evolutionary processing based on the Q-table can be improved.

[0076] Specifically, the above reward setting can be calculated based on the overall performance of the navigation points before and after execution, and the reward r is given by the difference in the objective function calculated before and after. When the objective function is better than the original objective function, the reward r will obtain a larger value. For example, the reward value r in the embodiment of the present invention can be set as follows: .

[0077] in, This represents the reward value obtained after the path update. This represents the position of the navigation point at the i-th location in the t-th generation. This represents the normalized objective function.

[0078] Furthermore, when the preset state is not updated, this embodiment of the invention can randomly initialize the Q-table. During the algorithm's execution, the Q-table is gradually updated using reward values. Updating the Q-table enables reinforcement learning, ultimately achieving constrained multi-objective processing.

[0079] Accordingly, the above step S42 operation can be specifically implemented by the following formula.

[0080] .

[0081] Among them, the above , These are the learning rate and the discount factor; It is a reward value; Indicates the current state and action; These represent the state and actions of the next generation after the update.

[0082] Furthermore, through the above-mentioned method, multicellular populations no longer rely solely on fixed genetic operators to generate offspring. Instead, they utilize the Q-learning method to balance exploration and exploitation. Based on the optimal distance state of the population or individuals, they dynamically select four influencing parameters: global exploration, local exploitation, and fast / slow convergence. This enables adaptive updates of the evolutionary strategy. Compared to the fixed genetic strategy used in related technologies, the embodiments of this invention are more adaptable to the needs of different evolutionary stages.

[0083] It's important to note that among multiple populations, single-celled populations often only consider constraints, shrinking the boundary only when all individuals satisfy the constraint boundary conditions. Multicellular populations, on the other hand, tend to comprehensively consider both constraints and the target population, adaptively adjusting boundary constraints. The incentive effect in the Q-table is the difference between the "target" value before and after the update. For single-celled populations, only the difference in constraint values ​​is considered, resulting in only two cases: "constraints satisfied, default difference value 0" and "constraints not satisfied, solution infeasible." This Q-incentive effect does not meet conventional usage requirements; therefore, a fixed constraint boundary reduction strategy is used for single-celled populations, while the Q-learning strategy is used only for multicellular populations.

[0084] Of course, in this embodiment of the invention, reinforcement learning methods such as Deep Q-Network (DQN) can also be used to update the first type of population. The method can be set according to actual needs, and this embodiment of the invention does not limit it.

[0085] Optionally, the value estimate of any of the navigation points is determined based on the following method: S61. After determining the state information of the navigation point, determine the value estimate of each action information corresponding to the navigation point based on the influence parameters and action operators corresponding to each action information.

[0086] The influencing parameters include inertia weight, a first factor, and a second factor; the inertia weight is positively correlated with the current speed of the navigation point, the first factor is positively correlated with memory capacity, and the second factor is positively correlated with interaction capacity; the action operator includes a first operator and a second operator, the first operator is negatively correlated with the number of iterations, and the second operator is negatively correlated with the first operator.

[0087] Optionally, during any update of the preset state table, the first operator is: ; The second operator is: ; Among them, the For the first operator, the The second operator; t is the iteration number; T max The preset iteration number threshold, the This is the preset smoothing coefficient.

[0088] Specifically, in Q-learning algorithms, actions often determine the most appropriate search behavior for a navigation point under different states. The general formula for calculating the Q-value is: .

[0089] It is evident that the definition of an action can be directly derived from inertia weights. Learning factors and Three influencing parameters determine this. Among them, the inertia weight... Determine the degree to which the current speed should be preserved. The larger the value, the better it maintains the original trajectory, and the stronger its global search capability. Conversely, the smaller the value, the stronger its local search capability. (Learning Factor) Reflecting the ability to remember historical states, The larger the value, the more likely the algorithm is to move towards its historical best position, resulting in better global search performance. (Learning factor) It represents the interaction capability between different navigation points, which drives the navigation points to converge toward the global optimal solution.

[0090] For example, embodiments of the present invention can define different action behaviors based on the impact of different parameter changes. For example, the following action behaviors can be set: global search, local exploration, slow convergence, and fast convergence, as shown in Table 1 below: Table 1:

[0091] Furthermore, in this embodiment of the invention, once the state information is determined, the Q-value for the corresponding situation can be calculated based on the influence parameters under different actions. By calculating the Q-values ​​under different actions, the optimal Q-value information can be obtained, and the optimal action can be selected. The influence parameters of a given action can be extracted, and the information of the new navigation point can be determined using the original navigation point and the influence parameters.

[0092] Specifically, in the action evolution of the Q-table, the action may get stuck in a local optimum due to the random setting of the initial value. Therefore, the first operator of the adaptive action operator is set. Second operator This gradually shifts the evolution of the Q-table from global exploration to convergence.

[0093] Optionally, the first operator and the second operator mentioned above can be respectively: .

[0094] .

[0095] Optionally, in this embodiment of the invention, setting the limit value to 2 can prevent the inertia weight reference coefficient from being too large due to excessively low expectations, and the learning factor from having too small an impact on the reference. By dynamically adjusting the reference coefficient, the Q-table action can focus more on exploring global diversity in the early stages and more on finding the optimal solution in the later stages.

[0096] For example, Figure 5 This is a schematic diagram of a preset state table provided in an embodiment of the present invention, such as... Figure 5As shown, this table can characterize the value estimates corresponding to state-action pairs, and the state information can be represented discretely. Actions can also be represented discretely according to actual needs.

[0097] Optionally, other perturbation state information can be added during the definition of the above state information. For example, the horizontal width and rate of change of the feasible domain of the current navigation point can be considered. If it is three-dimensional terrain, relevant information such as height gradient can also be considered. These can be set according to actual needs, and the embodiments of the present invention do not impose any restrictions on this.

[0098] This invention introduces a Q-learning mechanism in multicellular populations. It defines the state based on the distance difference between the navigation point and the individual optimal and global optimal states, distinguishes the performance differences under different states, and adaptively selects four actions: global search, local exploration, slow convergence, and fast convergence, thereby improving the convergence speed and global optimization capability.

[0099] It should be noted that current multi-agent robot path planning generally employs constrained multi-objective evolutionary algorithms for solving problems. Typical related technical solutions mainly include single-population or simple multi-population optimization methods based on NSGA-II, linear plug strategies with fixed constraint boundaries, and optimization frameworks with fixed evolutionary operator parameters. These methods can complete basic path planning tasks under conventional constraints, but they exhibit significant limitations when facing complex scenarios and multi-robot dynamic cooperative path planning problems with multiple constraints. For example, in agricultural inspection or emergency rescue scenarios involving dynamic obstacles, dynamic collision avoidance, path length and balance constraints, and multi-robot temporal cooperative constraints, traditional algorithms use a single population to simultaneously handle constraint satisfaction and objective optimization. This forces the population to simultaneously consider feasibility and optimality during the evolutionary process, leading to problems such as insufficient feasible region search, an excessively high proportion of infeasible solutions, and slow convergence speed. Furthermore, these technologies generally use fixed-parameter operators, which cannot adaptively switch between global search and local development modes based on the current evolutionary state of the population, iteration progress, and solution distribution characteristics. This makes it difficult for the algorithm to fully explore the search space in the early stages of evolution, and it is prone to getting trapped in local optima and premature convergence in the later stages. Furthermore, most existing constraint handling methods employ a fixed-rate boundary decay strategy. The shrinkage process of the constraint boundary is independent of the actual evolutionary state of the population, the proportion of feasible solutions, and the degree of constraint violation. For example, directly using strict constraint boundaries in the early stages of iteration can significantly compress the effective search range, while maintaining loose boundaries in the later stages of iteration can prevent the algorithm from quickly converging to the true feasible region. Ultimately, this leads to defects in path planning results such as tortuous paths, excessive length, and unsatisfied constraints. More importantly, existing technologies have almost never effectively integrated reinforcement learning with evolutionary algorithms, failing to achieve autonomous decision-making and dynamic adjustment of evolutionary strategies. When faced with complex and ever-changing path planning environments, they struggle to autonomously select the optimal search behavior, further reducing the optimization accuracy and stability of the algorithm under multiple constraints and multi-objective conditions.

[0100] Specifically, Figure 6 This is a schematic diagram of a path planning process provided by an embodiment of the present invention, such as... Figure 6 As shown in the embodiment of the present invention, the overall process is as follows: This mainly includes population initialization, dynamic boundary adjustment, population evolution, and termination conditions. First, two populations of size are randomly generated. population and ; then based on Assess the population and obtain and And initialize the Pareto file. Simultaneously initialize The Q-table information is then used; next, the maximum constraint violation for each constraint is set as the initial constraint boundary. and .

[0101] Then calculate the target population. Total constraint violations and fitness values ​​are dynamically adjusted. Constraint boundaries, initialize adaptive constraint scaling factors To determine whether to tighten or loosen the constraint. When When the value is greater than 0, it indicates that the performance of the current iteration solution has decreased, and the constraints should be relaxed appropriately to avoid invalid searches; when... When = 0, the current iterative solution is better than the previous generation, and the constraints should be gradually tightened to guide the population to approach the true Pareto front.

[0102] During dynamic boundary adjustment, constrain the population The primary focus is on the feasibility of the constraints, with the constraint boundaries shrinking faster than the target population. ; and the target population When solution performance degrades, constraints should be appropriately relaxed, with a primary focus on the convergence of the objective space. In each iteration, the constraint boundary update conditions are checked; if satisfied, the constraint boundaries for both the constraint-centric problem and the objective-centric problem are updated respectively. and .against Individuals at each navigation point obtained Calculate status information Query current action information .

[0103] The algorithm then enters the next evolutionary stage, constructing a new stage. and A binary tournament selection method was used to select participants from... and Select two parent populations and For population By simulating binary cross The generation of offspring population with the polynomial mutation operator For population The Q-table is used to learn operations to generate offspring. Then , Merge with the corresponding offspring population.

[0104] Then use the current evolutionary stage and Population assessment and The next generation of the population is obtained by selecting the execution environment. The reward is calculated based on the objective function values ​​before and after the update. Update the Q table. Finally, based on... Assessing new populations and The solution in the middle, update the Pareto archive The solution is found in the solution. Repeat the above steps until the maximum number of evaluations is reached; the algorithm then outputs the Pareto file. The solution in the solution is taken as the final result.

[0105] It is important to note that single-celled populations Focusing only on constraint satisfaction, a fixed constraint boundary contraction strategy is adopted; multicellular populations. Q-learning adaptively adjusts evolutionary parameters, balancing dynamic adjustment of constraint boundaries with convergence in the target space. These two aspects are closely linked and co-evolve, ultimately leading to the Pareto archive. The solution combines the advantages of both populations.

[0106] In summary, the embodiments of this invention address the shortcomings of related technologies in multi-robot dynamic cooperative constraint multi-objective path planning. By employing a reasonable population design, these embodiments balance constraint feasibility and objective optimality, improving the problem of insufficient feasible region search. Furthermore, these embodiments enable the algorithm to adaptively challenge the search strategy based on the population state, overcoming the convergence saturation and susceptibility to local optima caused by fixed parameters. They also achieve dynamic adaptive shrinkage of constraint boundaries, resulting in feasible optimal path outcomes under constrained conditions.

[0107] This invention divides the population into a unicellular population and a multicellular population. The unicellular population is solely driven by constraint satisfaction, while the multicellular population simultaneously considers both constraints and objective optimization, maintaining highly correlated constraint boundaries between the two populations. Through division of labor and cooperation, the unicellular population rapidly screens feasible solutions, while the multicellular population further optimizes the objective based on feasible solutions. These two processes operate in parallel; the unicellular population tends to focus on screening feasible solutions, while the multicellular population tends to focus on feasible solutions that optimize the objective. This mechanism addresses the inability of a single population to smooth the contradiction between constraints and objectives, improving the efficiency of feasible region search and the quality of solutions.

[0108] Furthermore, this invention addresses the issue that related technologies fail to incorporate reinforcement learning for autonomous decision-making in evolutionary strategies, leading to suboptimal results under complex conditions. These technologies rely on manually set evolutionary rules and cannot autonomously decide based on environmental changes, easily resulting in paths failing to meet conditions, insufficient searches, and suboptimal solutions in complex, multi-constraint scenarios. However, this invention utilizes a Q-learning mechanism to autonomously select the optimal evolutionary strategy through reinforcement learning. It maintains feasible, smooth, and optimal path planning under complex, multi-constraint conditions by designing Q-learning states and actions adapted to path planning and ensuring high correlation between the two populations.

[0109] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0110] This invention also provides a path planning device, see [link to relevant documentation]. Figure 7 The device 20 may include: Construction module 201 is used to construct at least two populations; each population contains a preset number of candidate paths for the target robot; The first acquisition module 202 is used to acquire the current evaluation result of the evolutionary guidance item corresponding to any of the populations; the evolutionary guidance items corresponding to different populations are different, and the evolutionary guidance item includes at least one of constraint violation degree and fitness. The first determining module 203 is used to determine the candidate paths whose evaluation results are not lower than other candidate paths as the target paths of the population based on the evaluation results of each population, and add the target paths to the candidate set; The boundary setting module 204 is used to set a constraint violation degree no lower than that of other candidate paths as the constraint boundary of the population to be shrunk based on the evaluation results of each candidate path in the population to be shrunk; the population to be shrunk is a population whose evolutionary guidance term includes the constraint violation degree; wherein, the constraint boundary of the first type of population is larger than the constraint boundary of the second type of population, the first type of population is a population that includes at least two evolutionary guidance terms, and the second type of population is a population that includes one evolutionary guidance term; Evolution module 205 is used to perform evolutionary processing on the candidate paths in the population based on the evaluation results and constraint boundaries of the population, so as to obtain a new population. The second determining module 206 is used to re-execute the operation of obtaining the current evaluation result of the evolutionary guidance item corresponding to each new population until the stopping condition is met, and then determine each target path contained in the candidate set as the final path.

[0111] Optionally, the above evolutionary module includes: The first selection submodule is used to select a parent population from the population based on the evaluation results and constraint boundaries of the population using a binary tournament selection method. The processing submodule is used to perform evolutionary processing based on the parent population to generate a child population. The merging submodule is used to merge the population with the offspring population to obtain the population to be selected. The second selection submodule is used to perform environmental selection on the population to be selected based on the constraint boundary to obtain a new population.

[0112] Optionally, the device further includes: The shrinking module is used to shrink the constraint boundary of the population to be shrunk in the following manner: ; Among them, the Let t represent the initial boundary of any candidate path with respect to the j-th constraint, and let t represent the iteration number. This represents the preset contraction rate, where j = 1, 2, ..., m, m is the number of constraints on the target robot, and T... max This is the preset threshold for the number of iterations.

[0113] Optionally, the device further includes: an update module for updating a preset state table; the preset state table includes value estimates for different state-action pairs; If the population is the first type of population, the evolution module is specifically used for: Based on the preset state table, evolutionary learning is performed on the parent population to generate the offspring population.

[0114] Optionally, the above-mentioned update module includes: The reward determination submodule is used to determine the reward value of each navigation point based on the difference in fitness of the navigation point after the path update and before the update. The value determination submodule is used to determine the current value estimate of the navigation point under the target state action pair based on the value estimate of the navigation point under the current state action pair, the reward value, the preset learning rate and discount factor, and the maximum value estimate of the navigation point under the target state action pair.

[0115] Optionally, the device further includes: a setting module for setting the state information in the state-action pair based on the following manner: For any given navigation point, calculate the state information of the navigation point based on the Euclidean distance between the navigation point and the globally optimal navigation point, the Euclidean distance between the navigation point and the individual historical optimal navigation point, and the obstacle factor; The state information is normalized to obtain discrete state information, and the state information in the preset state table is set as the discrete state information.

[0116] Optionally, the device further includes: The estimation and determination module determines the value estimate of any of the navigation points based on the following method: After determining the state information of the navigation point, the value estimate of each action information corresponding to the navigation point is determined based on the influence parameters and action operators corresponding to each action information. The influencing parameters include inertia weight, a first factor, and a second factor; the inertia weight is positively correlated with the current speed of the navigation point, the first factor is positively correlated with memory capacity, and the second factor is positively correlated with interaction capacity. The action operator includes a first operator and a second operator. The first operator is negatively correlated with the number of iterations, and the second operator is negatively correlated with the first operator.

[0117] Optionally, during any update of the preset state table, the first operator is: ; The second operator is: ; Among them, the For the first operator, the The second operator; t is the iteration number; T max The preset iteration number threshold, the This is the preset smoothing coefficient.

[0118] The present invention also provides an electronic device, see [link to relevant documentation]. Figure 8 The system includes: a processor 401, a memory 402, and a computer program 4021 stored in the memory and executable on the processor. When the processor executes the program, it implements the path planning method of the foregoing embodiments.

[0119] It should be noted that the electronic devices in the embodiments of the present invention include mobile electronic devices and non-mobile electronic devices.

[0120] The processor can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a FPGA (Field Programmable Gate Array), or other programmable devices, transistor logic devices, hardware components, or any combination thereof. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, or a combination of a DSP and a microprocessor.

[0121] The communication bus may include a path for transmitting information between the memory and the communication interface. The communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0122] The memory may be ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or it may be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disk Read Only), magnetic tape, floppy disk and optical data storage devices, etc.

[0123] The present invention also provides a readable storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the path planning method of the foregoing embodiments.

[0124] The present invention also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform... Figure 1 The path planning method shown.

[0125] The present invention also provides a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the various processes of the above-described path planning method embodiments, and achieving the same technical effect; to avoid repetition, further details are omitted here.

[0126] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0127] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid state disks (SSDs)).

[0128] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0129] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0130] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0131] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0132] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0133] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in fewer than all features of the single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0134] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0135] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention can also be implemented as a device or apparatus program for performing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0136] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0137] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0138] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0139] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A path planning method, characterized in that, The method includes: Construct at least two populations; each population contains a preset number of candidate paths for the target robot; For any of the populations, obtain the current evaluation result of the evolutionary guidance term corresponding to the population; different populations have different evolutionary guidance terms, and the evolutionary guidance term includes at least one of constraint violation degree and fitness. Based on the evaluation results of each population, the candidate paths whose evaluation results are not lower than other candidate paths are determined as the target paths of the population, and the target paths are added to the candidate set. Based on the evaluation results of each candidate path in the population to be shrunk, a constraint violation degree no lower than that of other candidate paths is set as the constraint boundary of the population to be shrunk; the population to be shrunk is a population whose evolutionary guidance term includes the constraint violation degree; wherein, the constraint boundary of the first type of population is larger than the constraint boundary of the second type of population, the first type of population is a population that includes at least two evolutionary guidance terms, and the second type of population is a population that includes one evolutionary guidance term. For the population, based on the evaluation results and constraint boundaries of the population, the candidate paths in the population are subjected to evolutionary processing to obtain a new population; For each new population, the operation of obtaining the current evaluation result of the evolutionary guidance item corresponding to the population is re-executed until the stopping condition is met, and then each target path contained in the candidate set is determined as the final path.

2. The method according to claim 1, characterized in that, Based on the evaluation results and constraint boundaries of the population, the candidate paths in the population are subjected to evolutionary processing to obtain a new population, including: Based on the evaluation results of the population and the constraint boundary, the parent population is selected from the population using the binary tournament selection method. Based on the parent population, an evolutionary process is performed to generate a child population; The population is merged with the offspring population to obtain the population to be selected. Based on the constraint boundary, environmental selection is performed on the candidate population to obtain a new population.

3. The method according to claim 2, characterized in that, Before selecting the parent population from the population using a binary tournament selection method based on the evaluation results and constraint boundaries of the population, the method further includes: The constraint boundary of the population to be shrunk is shrunk in the following manner: ； Among them, the Let t represent the initial boundary of any candidate path with respect to the j-th constraint, and let t represent the iteration number. This represents the preset contraction rate, where j = 1, 2, ..., m, m is the number of constraints on the target robot, and T... max This is the preset threshold for the number of iterations.

4. The method according to claim 2, characterized in that, After adding the target path to the candidate set, the method further includes: updating a preset state table; the preset state table includes value estimates for different state-action pairs; If the population is the first type of population, the step of generating a descendant population based on the evolutionary process of the parent population includes: Based on the preset state table, evolutionary learning is performed on the parent population to generate the offspring population.

5. The method according to claim 4, characterized in that, The update of the preset state table includes: For each navigation point, the reward value of the navigation point is determined based on the difference in fitness between the navigation point after the path update and before the update; Based on the value estimate of the navigation point under the current state action pair, the reward value, the preset learning rate and discount factor, and the maximum value estimate of the navigation point under the target state action pair, the current value estimate of the navigation point under the target state action pair is determined.

6. The method according to claim 4, characterized in that, The state information in the state-action pair is set based on the following method: For any given navigation point, calculate the state information of the navigation point based on the Euclidean distance between the navigation point and the globally optimal navigation point, the Euclidean distance between the navigation point and the individual historical optimal navigation point, and the obstacle factor; The state information is normalized to obtain discrete state information, and the state information in the preset state table is set as the discrete state information.

7. The method according to claim 5, characterized in that, The value estimate of any of the aforementioned navigation points is determined based on the following method: After determining the state information of the navigation point, the value estimate of each action information corresponding to the navigation point is determined based on the influence parameters and action operators corresponding to each action information. The influencing parameters include inertia weight, a first factor, and a second factor; the inertia weight is positively correlated with the current speed of the navigation point, the first factor reflects the ability to remember historical states, and the second factor represents the interaction ability between different navigation points. The action operator includes a first operator and a second operator. The first operator is negatively correlated with the number of iterations, and the second operator is negatively correlated with the first operator.

8. The method according to claim 7, characterized in that, During any update of the preset state table, the first operator is: ； The second operator is: ； Among them, the For the first operator, the The second operator; t is the iteration number; T max The preset iteration number threshold, the This is the preset smoothing coefficient.

9. A path planning device, characterized in that, The device includes: A construction module is used to construct at least two populations; each population contains a preset number of candidate paths for the target robot; The first acquisition module is used to acquire the current evaluation result of the evolutionary guidance item corresponding to any of the populations; the evolutionary guidance items corresponding to different populations are different, and the evolutionary guidance item includes at least one of constraint violation degree and fitness. The first determining module is used to determine the candidate paths whose evaluation results are not lower than other candidate paths as the target paths of the population based on the evaluation results of each population, and add the target paths to the candidate set; A boundary setting module is used to set a constraint violation degree no lower than that of other candidate paths as the constraint boundary of the population to be shrunk, based on the evaluation results of each candidate path in the population to be shrunk; the population to be shrunk is a population whose evolutionary guidance term includes the constraint violation degree; wherein, the constraint boundary of the first type of population is larger than the constraint boundary of the second type of population, the first type of population is a population including at least two evolutionary guidance terms, and the second type of population is a population including one evolutionary guidance term; An evolution module is used to perform evolutionary processing on candidate paths in the population based on the evaluation results and constraint boundaries of the population, so as to obtain a new population. The second determining module is used to re-execute the operation of obtaining the current evaluation result of the evolutionary guidance item corresponding to each new population until the stopping condition is met, and then determine each target path contained in the candidate set as the final path.

10. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program that, when executed by the processor, performs the method as described in any one of claims 1 to 8.

Images