Path planning method and robot

Abstract

The application is suitable for the technical field of robots, and provides a path planning method and a robot. The method comprises the following steps: acquiring environment information of a target area, establishing a grid map according to the environment information, the grid map comprising a start area, an end area and an obstacle area; determining a global path through a path search algorithm according to the grid map; and performing local path optimization through a dynamic window algorithm in the process of controlling the robot to move along the global path; wherein the dynamic window algorithm calculates the scores of each candidate local path through a trajectory cost function and determines an optimal local path, and the trajectory cost function introduces a strategy network output score obtained by predicting the candidate path through a pre-trained deep learning strategy model. The application can improve the adaptability of the robot in a dynamic environment.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of robot technology, and particularly relates to a path planning method and a robot. BACKGROUND

[0002] One of the core capabilities of mobile robots such as automated guided vehicles (AGV) is to autonomously reach a target point from a starting point. Path planning, which calculates a feasible motion trajectory to guide the robot to "how to move" in the environment, is the premise for the robot to get rid of manual remote control and realize autonomous decision-making. As a key technology in the AGV navigation system, the pros and cons of path planning are directly related to the running efficiency, safety and task completion quality of the whole vehicle.

[0003] Traditional path planning methods mainly include two categories: one is a graph search algorithm represented by A* algorithm and Dijkstra algorithm, which is suitable for global path planning in static environment; the other is a local obstacle avoidance method based on trajectory sampling and dynamic response such as dynamic window approach (DWA) and time elastic band (Teb), which emphasizes real-time reaction capability. However, in actual application, due to the fact that the environment usually contains dynamic targets, complex obstacles and unpredictable human-machine interference, a single method often shows deficiencies, such as: poor global path smoothness, many redundant nodes, affecting trajectory execution efficiency; local path planning is easy to fall into local optimum or appear trajectory oscillation; the perception mechanism is mostly passive response, and only path re-planning is performed after the obstacle appears; the path strategy lacks the ability to predict and actively adapt to the future environment dynamics. Therefore, a new path planning method is needed to effectively balance the global rationality and local dynamic obstacle avoidance capability of the path, improve the intelligent level of robot path planning and the stability of task execution, and improve its adaptability in dynamic environment. SUMMARY

[0004] Therefore, the embodiments of the present application provide a path planning method and a robot to improve the adaptability of the robot in dynamic environment.

[0005] The first aspect of the embodiments of the present application provides a path planning method, comprising:

[0006] obtain environment information of a target area, and establish a grid map according to the environment information, wherein the grid map contains a starting point area, an end point area and an obstacle area;

[0007] determine a global path by a path search algorithm according to the grid map;

[0008] During the movement of the robot along the global path, a local path is optimized by a dynamic window algorithm; wherein the dynamic window algorithm calculates scores of each candidate local path and determines an optimal local path by a trajectory cost function, and a policy network output score of the candidate path is obtained by introducing a pre-trained deep learning strategy model into the trajectory cost function.

[0009] In a possible implementation, the policy network output score of the candidate path obtained by the pre-trained deep learning strategy model comprises:

[0010] High-dimensional features of the grid map are extracted by a graph neural network to obtain high-dimensional features corresponding to the candidate path;

[0011] According to the high-dimensional features corresponding to the candidate path and the current state data of the robot, a state space vector of the deep learning strategy model is constructed and input into the deep learning strategy model, a candidate action distribution is generated by an Actor network, and a trajectory score value is output by a Critic network as the policy network output score.

[0012] In a possible implementation, the score result of the trajectory cost function is a weighted sum of a target orientation score, a speed maintenance score, an obstacle distance score, and the policy network output score.

[0013] In a possible implementation, when the dynamic window algorithm performs local path optimization, a minimum distance between the candidate local path and an obstacle region in the grid map is determined, and it is determined whether the minimum distance is less than a preset safety distance threshold;

[0014] If the minimum distance is less than the safety distance threshold, a penalty factor is determined according to the minimum distance, and the minimum distance is positively correlated with the size of the penalty factor;

[0015] According to the penalty factor, the speed of the robot is dynamically adjusted.

[0016] In a possible implementation, the calculation of the score of each candidate local path by the trajectory cost function comprises:

[0017] Initial weights of the target orientation score, the speed maintenance score, the obstacle distance score, and the policy network output score are obtained;

[0018] For each candidate local path, the size of each initial weight is dynamically adjusted according to the state data of the robot and the candidate local path.

[0019] In a possible implementation, the method further comprises:

[0020] performing time sequence occupancy prediction according to the historical sequence of the grid map to obtain a grid map prediction result in a future preset time period;

[0021] if the grid map prediction result shows that the path of the robot is occupied by an obstacle region in the future preset time period and the occupancy rate reaches a first preset threshold, re-performing the local path optimization by the dynamic window algorithm according to the grid map prediction result;

[0022] if the grid map prediction result shows that the path of the robot is occupied by an obstacle region in the future preset time period and the occupancy rate reaches a second preset threshold, re-performing the determination of the global path by the path search algorithm according to the grid map prediction result;

[0023] wherein the first preset threshold is less than the second preset threshold.

[0024] In a possible implementation, the obstacle region includes a static obstacle region and a dynamic obstacle region.

[0025] After the grid map is established according to the environment information, the method further includes:

[0026] updating the position of the dynamic obstacle region in real time according to the environment information.

[0027] In a possible implementation, the determination of the global path by the path search algorithm according to the grid map includes:

[0028] determining an initial global path by an A* path planning algorithm;

[0029] performing compression on the initial global path by a Floyd path compression algorithm;

[0030] performing smoothing on the compressed global path by a cubic B-spline curve.

[0031] In a possible implementation, the heuristic function of the A* path planning algorithm is a weighted sum of an actual cost from a starting point to a current node and a heuristic estimated cost from the current node to a target node, and the weight corresponding to the heuristic estimated cost from the current node to the target node is dynamically adjusted according to the search progress.

[0032] A second aspect of the embodiment of the application provides a robot, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the first aspect or any implementation manner of the first aspect when executing the computer program.

[0033] Compared with the prior art, the embodiment of the present application has the beneficial effects that:

[0034] (1) The embodiment of the present application adopts a hierarchical path planning structure, a global path is first determined by a path search algorithm to provide a clear target direction and task guidance, then a local path is optimized by a dynamic window algorithm during the movement of the robot along the global path, the target direction and obstacle avoidance safety are taken into account to improve the overall response capability of the system, thus the embodiment of the present application realizes the collaborative fusion of the global path planning algorithm and the local control optimization strategy, and realizes the effective balance between the global rationality of the path and the local dynamic obstacle avoidance capability.

[0035] (2) The embodiment of the present application introduces the policy network output score obtained by predicting the candidate path through the deep learning strategy model into the trajectory cost function of the dynamic window algorithm, the deep learning strategy model can exhibit good adaptive ability and long-term decision-making ability in a complex dynamic scene, thus it can learn the optimal path pattern from historical experience to improve the system robustness and task completion efficiency, and make up for the deficiency of the traditional dynamic window algorithm in the local optimization problem. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0037] Figure 1 is the overall implementation flowchart of the path planning method provided by an embodiment of the present application;

[0038] Figure 2 is the detailed implementation flowchart of the path planning method provided by an embodiment of the present application;

[0039] Figure 3 is the structure schematic diagram of the robot provided by the embodiment of the present application. DETAILED DESCRIPTION

[0040] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application, but these specific details are not intended to limit the present application. It should be apparent to those skilled in the art that the present application can be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary details.

[0041] In order to illustrate the technical solutions of the present application, the following will be described by specific examples.

[0042] The embodiment of the present application provides a path planning method, which fuses an improved A* algorithm and a DWA path planning of deep reinforcement learning, aims to improve the path rationality, execution stability and environment adaptability of an automatic guided vehicle (AGV) in a complex dynamic environment. The method fuses heuristic global search and strategy learning type local control by constructing a multi-level path planning system: firstly, an improved A algorithm based on introduction of a dynamic weighting mechanism and Floyd compression is used to complete global path generation, and a B spline curve is used to realize path smoothing. Secondly, a local path strategy network is designed based on a deep reinforcement learning PPO algorithm, so that the local obstacle avoidance capability and strategy intelligence in a dynamic environment are improved. In order to further enhance the environment interaction capability of the system, an active environment perception and information interaction mechanism is introduced, including semantic graph modeling, time sequence occupation prediction map, active perception strategy and perception-planning coupling mechanism. Among them, a high-dimensional topological graph is constructed by a graph neural network, a convolution LSTM is used to predict the future obstacle evolution trend, and a dynamic map feature is embedded into the strategy network to realize adaptive path planning. The method can dynamically identify obstacles and reconstruct the map, has the ability to actively explore unknown environments, integrates multi-source perception, path optimization and deep learning, has the advantages of clear structure, flexible strategy and strong environment adaptability, and is suitable for AGV path planning tasks in intelligent logistics, industrial storage and other high-dynamic complex scenes.

[0043] Figure 1 is a schematic diagram of the overall implementation process of the path planning method provided by an embodiment of the present application, and the following will be described in combination with Figure 1 , to illustrate the embodiment of the present application:

[0044] In step S101, the environment information of the target area is acquired, and a grid map is established according to the environment information. The grid map includes a start point area, an end point area and an obstacle area.

[0045] In this embodiment, a two-dimensional virtual environment is constructed based on the environment information, and the environment is divided into regular grid units by using a grid method, wherein each grid unit corresponds to a fixed physical space area, and the start point, the end point and the obstacle area that cannot be passed through are set in the map.

[0046] For example, in order to realize the calculability and environment adaptability of the path planning algorithm, a grid map model fusing static obstacle topology and dynamic obstacle identification is constructed in this embodiment.

[0047] Specifically, it includes:

[0048] Map gridding specification: the working area is divided into two-dimensional grids, and the size of each grid is .

[0049] The map contains five types of functional areas, the attributes of which are defined as follows:

[0050] (1) Passable area (white mark)

[0051] Attribute: Free grid not occupied by obstacles;

[0052] Function: As the search space and trajectory generation basis area for path planning algorithm;

[0053] Updating mechanism: Static attribute, remains unchanged after mapping.

[0054] (2) Static obstacle area (black mark)

[0055] Attribute: Unpassable area generated by fixed obstacle mapping;

[0056] Form of composition: Including straight wall, L-shaped structure, cross passage and other typical industrial scene obstacles;

[0057] Function: Simulate closed and semi-closed passage structure in real industrial environment;

[0058] Updating mechanism: Determined at initialization, remains unchanged after mapping;

[0059] Distribution requirement: Total proportion ≥ 20% of map area.

[0060] (3) Dynamic obstacle area (gray mark)

[0061] Attribute: Variable area occupied by real-time moving objects;

[0062] Function: Represent the real-time position of dynamic obstacles such as mobile robots and pedestrians;

[0063] Updating mechanism: Update the occupancy state dynamically at a frequency of 10Hz through multiple sensors.

[0064] (4) Start point area (red mark)

[0065] Attribute: Starting position of path planning task;

[0066] Setting specification: Clearly set grid coordinates at the initial mapping stage;

[0067] Updating mechanism: Changes with task setting.

[0068] (5) End point area (green mark)

[0069] Attribute: Target position of path planning task;

[0070] Setting specification: Clearly set grid coordinates at the initial mapping stage;

[0071] Update mechanism: change with task setting.

[0072] The embodiment generates a grid map, which is used to provide a basic search space for global path planning (such as improved A* algorithm), and also provides a decision input data structure for local path control (such as DWA algorithm) and reinforcement learning strategy (PPO). At the same time, the map data has the ability of real-time updating and visualization, and is suitable for task execution in a static and dynamic mixed environment. For example, the grid map can update the position of the dynamic obstacle area in real time according to the environment information.

[0073] In step S102, a global path is determined by a path search algorithm according to the grid map.

[0074] Here, a graph search algorithm such as A*, Dijkstra, etc. can be used for global path planning, which is not limited in the embodiment.

[0075] In the embodiment, the traditional A* path planning algorithm is optimized in multiple stages to construct a smooth global path generation mechanism for the execution layer, which specifically includes:

[0076] (1) Improved heuristic path search: Euclidean distance is used as the basic heuristic function, and a dynamic weighting mechanism is introduced to adjust the influence weight of the heuristic function in the search process, to improve the path guidance and global rationality. A minimum heap structure is used to construct a priority queue to optimize the node expansion order and search efficiency.

[0077] (2) Path redundant point compression processing: Floyd path compression algorithm is introduced to simplify unnecessary intermediate nodes in the original A* path, and only the path corner key points are reserved, which significantly reduces the path length and complexity, while preserving the overall connectivity and passability of the path.

[0078] (3) Path smoothing fitting processing: a cubic B-spline curve is used to interpolate and fit the compressed and optimized path point sequence to generate a continuous and derivable curve path. The constructed path has good curvature continuity, and the curvature constraint can be set according to the AGV kinematics model to ensure the executability and motion stability of the path on the physical level, adapt to the characteristics of non-omnidirectional wheeled platforms, and improve the running stability and comfort.

[0079] (4) Dynamic weighting mechanism: to improve the path guidance and search efficiency of the A* algorithm, the embodiment introduces a dynamic weight factor to adaptively adjust the influence of the heuristic cost term . The specific cost function is defined as follows:

[0080]

[0081] wherein: Total evaluation of node n; Actual cost from start to current node; Heuristic estimated cost (Euclidean distance) from current node to goal node; Dynamic weight factor, dynamically adjusted according to search progress.

[0082] Weight factor The design takes into account two factors: search progress ratio and weight update function.

[0083] Search progress ratio (progress ratio ):

[0084]

[0085] Weight update function:

[0086]

[0087] Wherein, Adjustment parameter , control the heuristic function more guiding in the early stage, and tend to be robust in the later stage. In the early stage of search tends to 0, at this time tends to , strong heuristic; in the late stage of search tends to 1, gradually returning to the standard A*. This mechanism essentially realizes the gradual strategy from heuristic dominance to balanced orientation, which helps to speed up the convergence of the early path and improve the overall rationality of the path.

[0088] Step S103, during the process of controlling the robot to move along the global path, local path optimization is performed through a dynamic window algorithm; wherein the dynamic window algorithm calculates the scores of each candidate local path through a trajectory cost function and determines the optimal local path, and the trajectory cost function introduces the policy network output score obtained by predicting the candidate path through the pre-trained deep learning strategy model.

[0089] DWA is a real-time local path planning and obstacle avoidance algorithm designed for mobile robots, especially suitable for real-time navigation in dynamic or unknown environments. It achieves the robot moving to the target point while avoiding obstacles and meeting motion constraints by filtering the optimal motion command in the "dynamic window" (feasible velocity set). The core logic of DWA is: under the current state of the robot, filter out the physically feasible and safe velocity set (dynamic window) through constraints, and then select the optimal velocity from the window through the evaluation function to drive the robot motion. In simple terms, the robot cannot directly plan a long path, but through "short-sighted" local decision-making - considering only possible movements in the next few seconds (based on current speed and acceleration limits), it ensures that each step is safe and moves closer to the target. This "local optimization + rolling execution" mode enables it to quickly respond to environmental changes (such as sudden obstacles).

[0090] The trajectory cost function of DWA is the "scoring standard" for selecting the optimal velocity from the dynamic window. The trajectory cost function of traditional DWA usually includes the following 3 core indicators: target orientation score, velocity maintenance score, and obstacle distance score. That is, the trajectory cost function is the weighted sum of the target orientation score, the velocity maintenance score, and the obstacle distance score.

[0091] This embodiment introduces the policy network output score of the deep learning strategy model into the trajectory cost function, uses the good adaptive ability and long-term decision-making ability of the deep learning strategy model in complex dynamic scenarios, learns the optimal path pattern from historical experience, improves system robustness and task completion efficiency, and makes up for the shortcomings of traditional DWA algorithm in local optimization. The specific mechanism is: for each candidate trajectory, extract its features; input into the trained deep learning strategy model to output the strategy score of the trajectory; jointly weighted and evaluated with traditional DWA evaluation indicators in the total score function; if there is a local trap or redundant path oscillation, the deep learning strategy model outputs a suggested fine-tuning target point position, or directly retreats and reselects the trajectory; finally, the motion command with the highest score is selected as the control output.

[0092] For example, the Proximal Policy Optimization (PPO) algorithm can be used, and the trajectory cost function can be described as:

[0093]

[0094] wherein,

[0095] : trajectory total score, used to select the optimal trajectory for execution;

[0096] , , , : Weighting coefficient, controls the weight of each score item;

[0097] : Trajectory angle score towards the goal (the closer to the goal direction, the higher the score);

[0098] : Minimum distance score between trajectory and obstacle (the farther from the safe distance, the higher the score);

[0099] : Trajectory terminal speed score (encourage faster path);

[0100] : Score predicted by the deep policy network (PPO) for this trajectory (represents the value evaluation of the trajectory in the training policy).

[0101] The expression is as follows:

[0102]

[0103]

[0104]

[0105]

[0106] wherein,

[0107] : Current trajectory angle (orientation angle at the end of the current trajectory);

[0108] : To the goal direction angle (direction angle from the current position to the target point);

[0109] : Maximum linear velocity allowed by the AGV;

[0110] : Minimum obstacle distance on the trajectory;

[0111] : Maximum evaluation distance, set to 2 meters;

[0112] : State feature corresponding to the candidate trajectory;

[0113] : State value network with parameter .

[0114] In one possible implementation, the weighting coefficient , , , The initial value of the weighting coefficient can be set by experience or simulation, for example: , , , In practical applications, the weighting coefficient can also be further adjusted by Bayesian optimization, for example, in a dynamic environment, the weight can be dynamically adjusted according to the current task state, for example, when approaching the target, the weight is increased , when the obstacles are dense, the weight is increased , and when the PPO confidence is high, the weight is increased , so as to improve the learning strategy dominance.

[0115] The embodiment of the application adopts a hierarchical path planning structure. Firstly, a path search algorithm determines a global path, and provides a clear target direction and task guidance. Then, during the movement of the robot along the global path, a local path is optimized by a dynamic window algorithm, taking into account the target orientation and obstacle avoidance safety, and improving the overall response capability of the system. Therefore, the embodiment of the application realizes the cooperative integration of the global path planning algorithm and the local control optimization strategy, and realizes the effective balance between the global rationality of the path and the local dynamic obstacle avoidance capability. In addition, the embodiment of the application introduces the strategy network output score obtained by predicting the candidate path through the deep learning strategy model into the trajectory cost function of the dynamic window algorithm. The deep learning strategy model can exhibit good adaptive ability and long-term decision-making ability in complex dynamic scenes, so it can learn the optimal path pattern from historical experience, improve the system robustness and task completion efficiency, and make up for the shortcomings of the traditional dynamic window algorithm in local optimization problems.

[0116] In some embodiments, step S102 comprises: determining an initial global path by an A* path planning algorithm; compressing the initial global path by a Floyd path compression algorithm; and smoothing the compressed global path by a cubic B-spline curve.

[0117] The details are explained as follows:

[0118] (1) Improved A* algorithm

[0119] The application designs an improved A algorithm combining a dynamic weighting mechanism and a minimum heap priority queue on the basis of a traditional A heuristic path search algorithm, which is used to generate a global path from a starting point to an end point in a discrete grid map.

[0120] The core idea of the A* algorithm is to prioritize the nodes to be expanded based on a cost function , and select the node with the minimum cost to expand, and the evaluation function is defined as follows:

[0121]

[0122] where,

[0123] : actual cost from start node to current node n.

[0124] : heuristic cost from current node n to goal node, calculated by Euclidean distance:

[0125]

[0126] : dynamic weight factor, dynamically adjusted according to search progress.

[0127] To enhance the directionality and efficiency of path search, especially to quickly approach the target area in the initial stage of search, the present application introduces a weighting strategy based on dynamic adjustment of search progress. The weight term is defined as:

[0128]

[0129] This mechanism realizes the smooth transition of heuristic function from dominance (initial stage) to balance (later stage), which not only improves the convergence speed of path, but also takes into account the stability and rationality of path. To further improve the efficiency of node expansion, the minimum heap structure is used as the priority queue, and the candidate nodes are automatically sorted according to value, ensuring that the current optimal expansion node is selected each time, and reducing the overall search time complexity.

[0130] (2) Floyd path optimization

[0131] Although the initial path generated by A* algorithm has connectivity and feasibility from the start node to the end node, since it is based on grid map for node-by-node search, the generated path often presents obvious polyline, containing a large number of intermediate nodes on the straight path, resulting in path not smooth, frequent turning, which is not conducive to subsequent trajectory smoothing processing and motion execution.

[0132] Therefore, the present application introduces Floyd path optimization algorithm to simplify the original A* path sequence, the steps are as follows:

[0133] Path segment feasibility detection: check any three points in the path , , in turn, if the start node can be directly connected to the end node without obstacle interference, the intermediate point is regarded as a redundant node and can be removed from the path;

[0134] Key point reservation mechanism: only reserve necessary turning points and direction changing points in the path, eliminate redundant nodes in the path that are consistent with the original direction.

[0135] The path compressed in this way not only significantly reduces the number of path nodes and reduces the planning complexity, but also has a more regular linear segment structure, which is beneficial to the subsequent B-spline curve fitting process.

[0136] The main role of Floyd optimization is to provide a structural basis for path smoothing. By reducing unnecessary polyline segments, continuous curves (such as cubic B-splines) can be generated on the basis of fewer control points to generate smooth, continuous and derivable execution paths. This optimization plays a role in the multi-stage path planning process, that is, it creates conditions for improving the geometric smoothness of the path while ensuring the feasibility of the path.

[0137] (3) B-spline curve path smoothing

[0138] After completing the Floyd path compression, the obtained path point sequence has greatly reduced redundant nodes, but still exists in the form of polyline segments, and the path presents discontinuous first derivative between nodes, with mutations at turning points, which is not conducive to the continuous and smooth execution of mobile robots.

[0139] To further improve the geometric continuity and execution feasibility of the path, this embodiment introduces a cubic B-spline curve to fit the path and realize the smoothing of the path.

[0140] The B-spline curve takes the path control points as input and generates a continuous and second-derivable curve trajectory on the entire path. Its definition is as follows:

[0141]

[0142] wherein, is the ith path control point; t is a parameterization variable defined in the node interval range; is the B-spline basis function, k is the order of the curve, k = 4 in this invention, and is recursively defined as follows:

[0143]

[0144]

[0145] Set the fixed node spacing and control points to generate a continuous and differentiable path curve, ensuring the motion continuity of the AGV execution process.

[0146] In the curve construction process, by setting equidistant or adaptive node parameters and endpoint constraints, the generated path is ensured to have:

[0147] Overall continuous smoothness: no sharp corners, meet the second-order geometric continuity;

[0148] Physical executability: the path is spatially continuous and smooth, meeting the needs of smooth motion of non-omni-directional wheeled robots;

[0149] Controllable curvature range: by adjusting the distribution of control points and the distance between nodes, the change of path curvature is limited to avoid trajectory oscillation.

[0150] The finally generated smooth path is not only more natural in space, but also more consistent with the actual kinematic constraints of AGV, improving the controllability, execution stability and navigation robustness of the path in the real physical platform.

[0151] In some embodiments, the step S103 of predicting the candidate path by the pre-trained deep learning strategy model to obtain a strategy network output score comprises:

[0152] (1) High-dimensional feature extraction of the grid map is performed by a graph neural network (GNN) to obtain high-dimensional features corresponding to the candidate path.

[0153] After the grid map is constructed, the graph neural network can be used to model the environmental topology, realize the fusion expression of the environmental structure, semantic category and dynamic trend. Mainly includes: constructing the topological atlas of obstacles, structures, and passing areas in the environment by the graph neural network, the nodes represent identifiable objects, and the edges describe their spatial or dynamic relationship, realizing high-dimensional modeling of the environmental structure and dynamic trend, and the extracted features are referenced by the path strategy network.

[0154] (2) According to the high-dimensional features corresponding to the candidate path and the current state data of the robot, a state space vector of the deep learning strategy model is constructed and input into the deep learning strategy model, and the Actor network generates a candidate action distribution, and the Critic network outputs a trajectory score value as the strategy network output score.

[0155] In order to improve the intelligence of path strategy decision, the PPO algorithm is introduced in this embodiment to establish a path strategy learning subsystem: state space: current position, target point, adjacent obstacle distribution, historical trajectory; action space: steering angle, linear velocity; reward function design: distance to target distance reduction (positive reward); close to obstacles (negative punishment); path smoothness (auxiliary reward); network structure adopts Actor-Critic double network design, and the training method is online simulation self-learning. After the strategy training is completed, it can be used for path node selection assistance, trajectory sampling scoring, local controller feedback optimization and other links, which significantly improves the path intelligence and generalization ability.

[0156] For the PPO algorithm:

[0157]

[0158] Strategy ratio, Dominance function : Trimming factor to prevent excessively large strategy updates.

[0159] The state space and action space are defined as follows:

[0160] Status: including current position, target point coordinates, obstacle distribution, path direction, etc.;

[0161] Actions: including steering angular velocity and linear velocity;

[0162] The reward function is designed by weighting the distance to the target, obstacle avoidance distance, and path smoothness. The reward function can be designed as follows:

[0163]

[0164] : The Euclidean distance from the AGV to the target point at the current moment;

[0165] : Distance (meters) to the nearest obstacle on the trajectory, when The task will terminate when triggered.

[0166] The change in the directional angle (in radians) between adjacent path points is used to penalize sharp turns.

[0167] Normalized speed;

[0168] Parameter settings: , , , , .

[0169] PPO uses an Actor-Critic architecture, generates path interactions with the help of a simulation environment, overcomes policy jitter by using a stable policy update method, and outputs a path policy network to guide the adjustment and planning of key nodes in the A* path.

[0170] In some embodiments, step S103, which calculates the score of each candidate local path using the trajectory cost function, includes:

[0171] Obtain the initial weights for the target orientation score, speed maintenance score, obstacle distance score, and policy network output score;

[0172] For each candidate local path, the size of each initial weight is dynamically adjusted according to the state data of the robot and the candidate local path.

[0173] In addition to expanding the dynamic window parameter (i.e., introducing the policy network output score), the following improvements are made to the DWA algorithm in the local control stage of the embodiment:

[0174] (1) Increase the sampling density to improve the trajectory accuracy.

[0175] The traditional DWA generates trajectories by sampling in the feasible velocity space (linear velocity and angular velocity ). The trajectory accuracy depends on the sampling resolution. In this embodiment, the sampling density in the velocity space is increased, and the ( , ) parameter pairs are more densely distributed; the time step accuracy of trajectory prediction is improved (the prediction period is reduced from 1s to 0.1s steps); each group of velocity pairs will generate a more detailed sequence of trajectory points, which is beneficial for obstacle detection and curvature constraint checking. This mechanism significantly improves the resolution and execution accuracy of the trajectory candidate set in complex environments, effectively improving obstacle avoidance sensitivity and navigation smoothness.

[0176] (2) Introduce a velocity dynamic adjustment mechanism to deal with sudden obstacles.

[0177] To adapt to temporary changes and uncertain disturbances of dynamic obstacles, a "velocity dynamic adjustment mechanism" is introduced to enable the AGV to automatically reduce the speed in the event of a sudden event and execute the deceleration steering strategy in advance:

[0178] The current speed is estimated by the end speed of the current trajectory :

[0179]

[0180] where is the current state position, is the position of the next time step, is the trajectory time step length.

[0181] When the minimum obstacle distance in the predicted trajectory (safety distance threshold), a speed penalty factor is introduced:

[0182]

[0183] where can be defined according to the obstacle distance as an exponential decreasing function:

[0184]

[0185] This is the distance penalty coefficient (adjusting the penalty rate), a constant of 1 / m, which controls how quickly the speed decreases as the obstacle approaches. This achieves an adaptive speed strategy of "slower as you get closer to the obstacle," improving obstacle avoidance robustness.

[0186] In some embodiments, a time dimension can be introduced on the basis of a traditional two-dimensional grid map, and temporal modeling methods such as convolutional LSTM can be used to predict the probability of obstacle distribution within a few future time steps, so as to realize trajectory prediction and early avoidance, and enhance path robustness and foresight.

[0187] That is: based on the historical sequence of the grid map, temporal occupancy prediction is performed to obtain the grid map prediction results for the future preset time period; if the grid map prediction results show that the robot's path is occupied by obstacle areas within the future preset time period, and the occupancy rate reaches the first preset threshold, then the local path optimization is re-executed using the dynamic window algorithm based on the grid map prediction results; if the grid map prediction results show that the robot's path is occupied by obstacle areas within the future preset time period, and the occupancy rate reaches the second preset threshold, then the global path is re-executed using the path search algorithm based on the grid map prediction results; wherein, the first preset threshold is less than the second preset threshold.

[0188] For example: using convolutional LSTM to process historical raster map sequences Perform time-series occupancy prediction; if future Probability of critical path points being occupied within s At 0.7s, local path resampling (local replanning) is triggered; if prediction is performed for 5 consecutive frames... At 0.9s, global A* path replanning and subsequent local path replanning are triggered.

[0189] Figure 2 This is a detailed implementation flowchart of a path planning method provided in an embodiment of the present invention. The following is a combined description... Figure 2 The following is a description of this embodiment:

[0190] Step 1: Environmental modeling and map building.

[0191] Construct a simulation platform with a size of A two-dimensional simulation environment. To facilitate path planning and dynamic obstacle avoidance, the entire environment is discretized using a standard grid map method. The equidistant grid is divided into 400 grid cells, which has spatial discretization and numerical computation basis.

[0192] The obstacles in the environment are designed in a structured manner, mainly including:

[0193] Cross-shaped wall structure: simulating partitions or transmission channels in industrial scenarios;

[0194] Two irregular L-shaped obstacles: representing obstacle forms such as corner shelves, equipment boundaries, etc.

[0195] The overall obstacle ratio is controlled at about 20%, ensuring that path planning is challenging while leaving enough free space for multi-strategy testing.

[0196] The starting point coordinates are set to (2, 2), the end point coordinates are set to (18, 18), and the passable area is white grid, and the impassable area (obstacle) is marked as black.

[0197] During the map construction process, visualization and verification mechanisms are integrated: use tools to render the environment map, and display the grid state in real time; introduce reachability verification module to verify the connectivity from the starting point to the end point, to ensure that there is at least one passageway; provide API interface for path search module, DWA controller and deep policy network to call map structure; provide unified and standardized data foundation for subsequent global path planning and local obstacle avoidance control. The map construction method combines structured obstacle layout and randomness, which helps to test the robustness and adaptability of path planning algorithm under multi-obstacle interference in complex industrial or logistics environment.

[0198] Step two, environment topology modeling based on graph neural network.

[0199] Node generation: cluster the obstacle regions marked in the static grid map, and each independent obstacle or structural region is regarded as a node. Node attributes include: center position (coordinates); geometric type (such as straight line, L-shaped, etc.); semantic category (wall, shelf, boundary, etc.); dynamic label (whether covered or approached by dynamic obstacles).

[0200] Edge construction and feature calculation: if the Euclidean distance between two obstacle nodes is less than a threshold (such as 2 meters), then an undirected edge is established between them, and the edge attributes include: spatial distance; environmental semantic similarity; dynamic interaction frequency (based on historical motion data statistics).

[0201] Graph neural network inference and vector embedding: after constructing the graph structure, use the graph convolutional network (GCN) structure to perform propagation calculation on the graph, extract the structural context features of each node, and form the embedding vector.

[0202] The output features are used as the input state vector of PPO, which is transmitted to the PPO policy network to realize the perception-strategy fusion mechanism. This topology mapping module not only improves the understanding of the geometric constraints of the environment for path planning, but also provides structural prior knowledge for subsequent time series prediction and strategy scoring.

[0203] Step three, determine the initial global path by A* path planning algorithm.

[0204] Step four, compress the initial global path by Floyd path compression algorithm.

[0205] Step five, smooth the compressed global path by cubic B-spline curve.

[0206] Step six, perform local optimization by DWA algorithm, extract the state sequence features of each candidate trajectory , input them into the trained PPO policy network Actor part, output the policy score of the trajectory , and jointly weight the evaluation indicators in the total score function; if there is a local trap or redundant path repetition, the PPO outputs a suggested fine-tuning target point position, or directly retreats and reselects the trajectory; finally, the motion command with the highest score is selected as the control output.

[0207] Step seven, select the candidate local path with the highest score as the optimal local path.

[0208] Step eight, use convolution LSTM to perform time series occupancy prediction on the historical grid map sequence ; if the key path point occupancy probability 0.7s in the future , trigger local path resampling (local replanning); if the predicted 0.9s for 5 consecutive frames, trigger global A* path replanning.

[0209] The innovations of the present application include:

[0210] (1) Introduce time dimension on the basis of traditional two-dimensional grid map, use convolution LSTM and other time series modeling methods to predict the obstacle distribution probability in the future time steps, realize trajectory prediction and early avoidance, and enhance the robustness and foresight of the path.

[0211] (2) The structural features generated by the graph neural network are used as the state embedding vector of the path strategy network, participating in the trajectory sampling, scoring and policy updating process, realizing the deep coupling from environmental structure perception to path strategy output.

[0212] (3) Introduce PPO reinforcement learning mechanism, train the policy network to learn path node selection rules and trajectory quality evaluation criteria. The policy network exhibits good adaptive ability and long-term decision-making ability in complex dynamic scenarios, can learn the optimal path pattern from historical experience, improve the robustness and task completion efficiency of the system, and make up for the shortcomings of traditional DWA algorithm in local optimization problem.

[0213] ​(4) Adopt hierarchical path planning structure: the global path planning module generates the global navigation path from the starting point to the end point based on the improved A* algorithm, and provides clear target direction and task guidance; the local path planning module adopts the DWA algorithm, dynamically samples feasible trajectories according to the current robot state and the perceived environment on the basis of the global path guidance, evaluates and selects the optimal control command in real time, and gives consideration to target orientation and obstacle avoidance safety, thereby improving the overall response capability of the system.

[0214] (5) During path execution, the system has the ability of active information selection, which can actively obtain the information of the environment region that is currently invisible or uncertain through adjusting the sensor viewing angle, path offset, bypassing the blocked region and the like. The mechanism is obtained through reinforcement learning strategy training, and greatly enhances the perception ability of the system to dynamic changes and the environmental adaptability.

[0215] (6) The method upgrades the path planning system from the traditional "passive perception-path response" mode to the "structure mapping-dynamic prediction-active interaction-strategy control" closed-loop mechanism, realizes the deep integration of global rationality, local flexibility and environmental adaptability of path planning. The mechanism has significant improvement in the success rate of navigation, trajectory continuity and system robustness in a dynamic environment with multiple obstacles.

[0216] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0217] Figure 3 is a schematic diagram of a robot provided by an embodiment of the present application. As shown in Figure 3 , the robot 3 of the embodiment includes a processor 30 and a memory 31. The memory 31 stores a computer program 32. The processor 30 implements the steps in each of the above method embodiments when executing the computer program 32. Alternatively, the processor 30 implements the functions of each module / unit in the above device embodiments when executing the computer program 32.

[0218] For example, the computer program 32 can be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 30 to complete the present application. The one or more modules / units can be a series of computer program instruction segments that can complete a specific function, which are used to describe the execution process of the computer program 32 in the robot 3.

[0219] The robot 3 can include, but is not limited to, the processor 30, the memory 31. Those skilled in the art can understand, Figure 3The example of the robot 3 is merely illustrative and does not limit the robot 3, which can include more or fewer components than shown, or combine some components, or have different components, such as the robot 3 can also include an input / output device, a network access device, a bus, etc.

[0220] The processor 30 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0221] The memory 31 can be an internal storage unit of the robot 3, such as a hard disk or a memory of the robot 3. The memory 31 can also be an external storage device of the robot 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the robot 3. Further, the memory 31 can also include both the internal storage unit and the external storage device of the robot 3. The memory 31 is used to store the computer program 32 and other programs and data required by the robot 3. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0222] For the convenience and brevity of description, only the division of the above functional modules / units is exemplified, and in actual application, the above functions can be completed by different functional modules / units according to needs. The above modules / units can be realized in the form of hardware, in the form of software, or in the form of combination of hardware and software.

[0223] The embodiment of the application also provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method in each method embodiment described above is implemented.

[0224] The embodiment of the application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method in each method embodiment described above is implemented.

[0225] The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0226] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments. If there is no special description and logical conflict, the terms and / or descriptions of different embodiments are consistent and can be mutually referred to, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0227] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A path planning method characterized by, The method comprises: obtaining environment information of a target area, and establishing a grid map according to the environment information, wherein the grid map comprises a start area, an end area and an obstacle area; determining a global path by a path search algorithm according to the grid map; performing local path optimization by a dynamic window algorithm during movement of a robot along the global path; wherein the dynamic window algorithm calculates scores of each candidate local path by a trajectory cost function and determines an optimal local path, and the trajectory cost function introduces a strategy network output score obtained by predicting a candidate path by a pre-trained deep learning strategy model; a score result of the trajectory cost function is a weighted sum of a target orientation score, a speed maintenance score, an obstacle distance score and the strategy network output score; when performing local path optimization, the dynamic window algorithm determines a minimum distance between the candidate local path and the obstacle area in the grid map, and judges whether the minimum distance is less than a preset safety distance threshold; if the minimum distance is less than the safety distance threshold, a penalty factor is determined according to the minimum distance, and the minimum distance and the penalty factor are positively correlated in size; and the speed of the robot is dynamically adjusted according to the penalty factor; calculating the scores of each candidate local path by the trajectory cost function comprises: obtaining initial weights of the target orientation score, the speed maintenance score, the obstacle distance score and the strategy network output score; for each candidate local path, dynamically adjusting the size of each initial weight according to state data of the robot and the candidate local path; wherein the initial weight of the target orientation score increases when approaching the target; the initial weight of the speed maintenance score is positively correlated with obstacle density; and the initial weight of the strategy network output score is positively correlated with confidence; determining the global path by the path search algorithm according to the grid map comprises: determining an initial global path by an A* path planning algorithm; compressing the initial global path by a Floyd path compression algorithm; smoothing the compressed global path by a cubic B-spline curve; a heuristic function of the A* path planning algorithm is a weighted sum of an actual cost from a start point to a current node and a heuristic estimated cost from the current node to a target node; and a weight corresponding to the heuristic estimated cost from the current node to the target node is dynamically adjusted according to search progress: ； ； wherein, is a search progress ratio; is an actual cost from the start node to the current node; is a heuristic estimated cost from the current node to the goal node; is a dynamic weight factor; is a tuning parameter.

2. The path planning method of claim 1, wherein, predicting the candidate path by the pre-trained deep learning strategy model to obtain the strategy network output score comprises: extracting high-dimensional features of the grid map by a graph neural network to obtain high-dimensional features corresponding to the candidate path; constructing a state space vector of the deep learning strategy model according to the high-dimensional features corresponding to the candidate path and current state data of the robot, and inputting the state space vector into the deep learning strategy model, so that an actor network generates a candidate action distribution, and a critic network outputs a trajectory score value as the strategy network output score.

3. The path planning method according to claim 1 or 2, characterized in that, The method further comprises: According to a historical sequence of the grid map, a time sequence occupancy prediction is performed to obtain a grid map prediction result in a future preset time period; If the grid map prediction result shows that the path of the robot is occupied by an obstacle region in the future preset time period, and the occupancy rate reaches a first preset threshold, then according to the grid map prediction result, the local path optimization by the dynamic window algorithm is re-executed; If the grid map prediction result shows that the path of the robot is occupied by an obstacle region in the future preset time period, and the occupancy rate reaches a second preset threshold, then according to the grid map prediction result, the global path determined by the path search algorithm is re-executed; Wherein, the first preset threshold is less than the second preset threshold.

4. The path planning method according to claim 1 or 2, characterized by, The obstacle region includes a static obstacle region and a dynamic obstacle region; After the grid map is established according to the environment information, the method further comprises: According to the environment information, the position of the dynamic obstacle region is updated in real time.

5. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the method of any one of claims 1 to 4.

Images