Path planning method and robot

Abstract

The invention is suitable for the technical field of robots, and provides a path planning method and a robot, and the method comprises the steps: obtaining the environment information of a target area, building a grid map according to the environment information, and enabling the grid map to comprise a starting point area, an end point area and an obstacle area; determining a global path through a path search algorithm according to the grid map; in the process of controlling the robot to move along the global path, performing local path optimization through a dynamic window algorithm; wherein the dynamic window algorithm calculates the score of each candidate local path through a trajectory cost function and determines the optimal local path, and a strategy network output score obtained by predicting the candidate paths through a pre-trained deep learning strategy model is introduced into the trajectory cost function. The adaptive capacity of the robot in a dynamic environment can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of robots, and in particular relates to a path planning method and a robot. Background Art

[0002] One of the core capabilities of mobile robots, such as automated guided vehicles (AGVs), is the ability to autonomously navigate from a starting point to a destination. Path planning, which calculates a feasible trajectory to guide the robot's movement within its environment, is a prerequisite for robots to achieve autonomous decision-making, freeing them from manual remote control. As a key technology in AGV navigation systems, path planning's performance directly impacts the overall vehicle's operational efficiency, safety, and mission performance.

[0003] Traditional path planning methods fall into two main categories: graph search algorithms, such as the A* algorithm and the Dijkstra algorithm, are suitable for global path planning in static environments. Local obstacle avoidance methods, such as the Dynamic Window Approach (DWA) and the Time Elastic Band (Teb), rely on trajectory sampling and dynamic response, emphasizing real-time responsiveness. However, in practical applications, where environments often contain dynamic targets, complex obstacles, and unpredictable human-robot interference, single approaches often exhibit shortcomings. These include poor global path smoothness and excessive node redundancy, which affect trajectory execution efficiency; local path planning is prone to falling into local optima or trajectory oscillation; perception mechanisms are often passive, requiring path replanning only after obstacles emerge; and path strategies lack the ability to predict and proactively adapt to future environmental dynamics. Therefore, a new path planning method is needed that effectively balances global path rationality with local dynamic obstacle avoidance capabilities, thereby improving the intelligence of robot path planning, the stability of task execution, and its adaptability in dynamic environments. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a path planning method and a robot to improve the adaptability of the robot in a dynamic environment.

[0005] A first aspect of an embodiment of the present invention provides a path planning method, including: Acquiring environmental information of a target area, and establishing a grid map based on the environmental information, wherein the grid map includes a starting area, an end area, and an obstacle area; Determining a global path using a path search algorithm based on the grid map; In the process of controlling the robot to move along the global path, local path optimization is performed using a dynamic window algorithm; wherein, the dynamic window algorithm calculates the score of each candidate local path and determines the optimal local path through a trajectory cost function, and the trajectory cost function introduces the strategy network output score obtained by predicting the candidate path through a pre-trained deep learning strategy model.

[0006] In one possible implementation, a pre-trained deep learning policy model is used to predict candidate paths and obtain a policy network output score, including: Extract high-dimensional features from the grid map using a graph neural network to obtain high-dimensional features corresponding to the candidate paths; Based on the high-dimensional features corresponding to the candidate paths and the current state data of the robot, the state space vector of the deep learning strategy model is constructed and input into the deep learning strategy model. The actor network generates the candidate action distribution, and the critic network outputs the trajectory score value as the output score of the strategy network.

[0007] In a possible implementation, the scoring result of the trajectory cost function is a weighted sum of a target orientation score, a speed maintenance score, an obstacle distance score, and an output score of the policy network.

[0008] In a possible implementation, when performing local path optimization, the dynamic window algorithm determines the minimum distance between the candidate local path and the obstacle area in the grid map, and determines whether the minimum distance is less than a preset safety distance threshold; If the minimum distance is less than the safety distance threshold, a penalty factor is determined according to the minimum distance, and the minimum distance is positively correlated with the size of the penalty factor; The speed of the robot is dynamically adjusted according to the penalty factor.

[0009] In a possible implementation, calculating the score of each candidate local path using a trajectory cost function includes: Obtaining initial weights of the target orientation score, the speed maintenance score, the obstacle distance score, and the policy network output score; For each candidate partial path, the size of each initial weight is dynamically adjusted according to the state data of the robot and the candidate partial path.

[0010] In a possible implementation, the method further includes: Perform time series occupancy prediction based on the historical sequence of the grid map to obtain grid map prediction results within a preset time period in the future; If the grid map prediction result shows that the robot's path is occupied by an obstacle area within the future preset time period, and the occupancy rate reaches a first preset threshold, then re-execute the local path optimization using the dynamic window algorithm based on the grid map prediction result; If the grid map prediction result shows that the robot's path is occupied by an obstacle area within the future preset time period, and the occupancy rate reaches a second preset threshold, re-executing the determination of the global path by the path search algorithm based on the grid map prediction result; The first preset threshold is smaller than the second preset threshold.

[0011] In a possible implementation, the obstacle area includes: a static obstacle area and a dynamic obstacle area; After establishing the grid map according to the environmental information, the method further includes: The position of the dynamic obstacle area is updated in real time according to the environmental information.

[0012] In a possible implementation, determining the global path by a path search algorithm based on the grid map includes: Determine the initial global path through the A* path planning algorithm; Compress the initial global path using the Floyd path compression algorithm; The compressed global path is smoothed using a cubic B-spline curve.

[0013] In one possible implementation, the heuristic function of the A* path planning algorithm is the weighted sum of the actual cost from the starting point to the current node and the heuristic estimated cost from the current node to the target node; and the weight corresponding to the heuristic estimated cost from the current node to the target node is dynamically adjusted according to the search progress.

[0014] A second aspect of an embodiment of the present invention provides a robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the robot implements the steps of the first aspect or any one of the implementation methods of the first aspect.

[0015] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: (1) The embodiment of the present invention adopts a hierarchical path planning structure. First, the path search algorithm determines the global path, providing a clear target direction and task guidance. Then, in the process of controlling the robot to move along the global path, the local path optimization is performed through the dynamic window algorithm, taking into account both the target orientation and the obstacle avoidance safety, thereby improving the overall response capability of the system. Therefore, the embodiment of the present invention realizes the coordinated integration of the global path planning algorithm and the local control optimization strategy, and achieves an effective balance between the global rationality of the path and the local dynamic obstacle avoidance capability.

[0016] (2) The embodiment of the present invention introduces the strategy network output score obtained by predicting the candidate path through the deep learning strategy model into the trajectory cost function of the dynamic window algorithm. The deep learning strategy model can demonstrate good adaptive ability and long-term decision-making ability in complex dynamic scenarios. Therefore, it can learn the optimal path pattern from historical experience, improve the system robustness and task completion efficiency, and make up for the shortcomings of the traditional dynamic window algorithm in the local optimal problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a schematic diagram of the overall implementation process of the path planning method provided by one embodiment of the present invention; Figure 2 Detailed implementation flow diagram of a path planning method provided by an embodiment of the present invention; Figure 3 Schematic diagram of the structure of the robot provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0019] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0020] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0021] Embodiments of the present invention provide a path planning method that integrates an improved A* algorithm with deep reinforcement learning-based DWA path planning, aiming to improve the path rationality, execution smoothness, and environmental adaptability of automated guided vehicles (AGVs) in complex dynamic environments. This method constructs a multi-level path planning system, integrating heuristic global search with policy-learning-based local control. First, a global path is generated using an improved A* algorithm that incorporates a dynamic weighting mechanism and Floyd compression, and path smoothing is achieved using B-spline curves. Second, a local path policy network is designed based on the deep reinforcement learning PPO algorithm to enhance local obstacle avoidance and policy intelligence in dynamic environments. To further enhance the system's environmental interaction capabilities, active environmental perception and information interaction mechanisms are introduced, including semantic graph modeling, a temporal occupancy prediction map, an active perception policy, and a perception-planning coupling mechanism. A graph neural network is used to construct a high-dimensional topological map. A convolutional LSTM is used to predict future obstacle evolution trends. The policy network embeds dynamic map features to achieve adaptive path planning. This method can dynamically identify obstacles and reconstruct maps, has the ability to actively explore unknown environments, integrates multi-source perception, path optimization and deep learning, and has the advantages of clear structure, flexible strategy and strong environmental adaptability. It is suitable for AGV path planning tasks in highly dynamic and complex scenarios such as intelligent logistics and industrial warehousing.

[0022] Figure 1 This is a schematic diagram of the overall implementation process of the path planning method provided by an embodiment of the present invention. Figure 1 , the embodiments of the present invention are described: Step S101 : Acquire environmental information of a target area, and create a grid map based on the environmental information. The grid map includes a starting area, an end area, and an obstacle area.

[0023] This embodiment constructs a two-dimensional virtual environment based on environmental information and divides the environment into regular grid units using a grid method, where each grid unit corresponds to a fixed physical space area, and a starting point, an end point, and an inaccessible obstacle area are set in the map.

[0024] For example, to achieve computability and environmental adaptability of the path planning algorithm, this embodiment constructs a grid map model that integrates static obstacle topology and dynamic obstacle identification.

[0025] Specifically include: Map rasterization specification: Divide the working area into A two-dimensional grid with each grid size of .

[0026] The map contains five types of functional areas, whose attributes are defined as follows: (1) Passable area (white mark) Attributes: Free grid not occupied by obstacles; Function: Serves as the search space for path planning algorithms and the basic area for trajectory generation; Update mechanism: static properties, remain unchanged after the map is built.

[0027] (2) Static obstacle area (black mark) Attributes: Impassable areas generated by fixed obstacle mapping; Composition form: including straight walls, L-shaped structures, cross passages and other typical industrial scene obstacles; Function: Simulate closed and semi-closed channel structures in real industrial environments; Update mechanism: determined during initialization and remains unchanged after the map is built; Distribution requirements: total proportion ≥ 20% of the map area.

[0028] (3) Dynamic obstacle area (gray mark) Attributes: variable area occupied by real-time moving objects; Function: Characterize the real-time position of dynamic obstacles such as mobile robots and pedestrians; Update mechanism: Dynamically updates occupancy status at 10Hz through multiple sensors.

[0029] (4) Starting area (red mark) Attributes: starting position of the path planning task; Setting standards: clearly set the grid coordinates at the beginning of map construction; Update mechanism: changes with task settings.

[0030] (5) Finish area (green mark) Attributes: target location of the path planning task; Setting standards: clearly set the grid coordinates at the beginning of map construction; Update mechanism: changes with task settings.

[0031] This embodiment generates a grid map, which provides the basic search space for global path planning (such as the modified A* algorithm) and the decision input data structure for local path control (such as the DWA algorithm) and reinforcement learning strategies (PPO). Furthermore, the map data features real-time updates and visualization capabilities, making it suitable for executing tasks in mixed static and dynamic environments. For example, the grid map can update the location of dynamic obstacle areas in real time based on environmental information.

[0032] Step S102: determining a global path through a path search algorithm according to the grid map.

[0033] Here, a graph search algorithm represented by A*, Dijkstra, etc. may be used to perform global path planning, which is not specifically limited in this embodiment.

[0034] In this embodiment, the traditional A* path planning algorithm is optimized in multiple stages to construct a smooth global path generation mechanism for the execution layer, specifically including: (1) Improvement of heuristic path search: Euclidean distance is used as the basic heuristic function, and a dynamic weighting mechanism is introduced to adjust the influence weight of the heuristic function in the search process, thereby improving path guidance and global rationality. A minimum heap structure is used to construct a priority queue, optimizing the node expansion order and search efficiency.

[0035] (2) Path redundancy point compression processing: The Floyd path compression algorithm is introduced to simplify the non-essential intermediate nodes in the original A* path, retaining only the key points of the path corners, significantly reducing the path length and complexity while retaining the overall connectivity and passability of the path.

[0036] (3) Path smoothing fitting: A cubic B-spline curve is used to interpolate and fit the compressed and optimized path point sequence to generate a continuous and differentiable curved path. The constructed path has good curvature continuity and can be set with curvature constraints according to the AGV kinematic model to ensure the path's physical feasibility and motion stability, adapt to the characteristics of non-omnidirectional wheeled platforms, and improve operational smoothness and comfort.

[0037] (4) Dynamic weighting mechanism: In order to improve the path guidance and search efficiency of the A* algorithm, this embodiment introduces a dynamic weight factor based on the traditional heuristic function. , used to adaptively adjust the heuristic cost term The specific cost function is defined as follows:

[0038] in: is the total valuation of node n; is the actual cost from the starting point to the current node; Estimate the heuristic cost (Euclidean distance) from the current node to the target node; It is a dynamic weight factor, which is adjusted dynamically according to the search progress.

[0039] Weighting Factor The design of considers two factors: search progress ratio and weight update function.

[0040] Search progress ratio (progress ratio ):

[0041] Weight update function:

[0042] in, To adjust the parameters , the control heuristic function is more guiding in the early stage and tends to be more robust in the later stage. tends to 0, at this time tend to , highly inspiring; at the end of the search As the probability of the path converges to 1, it gradually returns to standard A*. This mechanism essentially implements a gradual transition from a heuristic-led to a balanced-led strategy, which helps accelerate the early convergence of the path and improves the overall rationality of the path.

[0043] In step S103, while controlling the robot to move along the global path, local path optimization is performed using a dynamic window algorithm. The dynamic window algorithm calculates the score of each candidate local path and determines the optimal local path using a trajectory cost function, and the trajectory cost function includes a strategy network output score obtained by predicting the candidate path using a pre-trained deep learning strategy model.

[0044] DWA is a real-time local path planning and obstacle avoidance algorithm designed specifically for mobile robots, particularly suited for real-time navigation in dynamic or unknown environments. It selects the optimal motion instructions within a "dynamic window" (a set of feasible velocities) to enable the robot to avoid obstacles and move toward its target while satisfying motion constraints. DWA's core logic involves using constraints to select a set of physically feasible and safe velocities (a dynamic window) within the robot's current state. An evaluation function is then used to select the optimal velocity from this window to drive the robot's motion. Simply put, the robot cannot directly plan long paths. Instead, it makes short-sighted, local decisions, considering only possible movements within the next few seconds (based on current speed and acceleration constraints) to ensure each step is safe and leads toward the target. This "local optimization + rolling execution" model enables rapid response to environmental changes, such as the sudden appearance of obstacles.

[0045] The DWA trajectory cost function is a scoring criterion for selecting the optimal speed within a dynamic window. Traditional DWA trajectory cost functions typically include three core metrics: target orientation score, speed maintenance score, and obstacle distance score. Specifically, the trajectory cost function is a weighted sum of these three scores.

[0046] This embodiment incorporates the policy network output score of a deep learning policy model into the trajectory cost function. Leveraging the deep learning policy model's excellent adaptability and long-term decision-making capabilities in complex dynamic scenarios, it learns optimal path patterns from historical experience, improving system robustness and task completion efficiency while addressing the shortcomings of traditional DWA algorithms in addressing local optimality. The specific mechanism is as follows: For each candidate trajectory, its features are extracted; these are input into the trained deep learning policy model, which outputs a policy score for that trajectory; this score is weighted and evaluated alongside traditional DWA evaluation metrics in the overall scoring function; if local traps or redundant paths repeatedly oscillate, the deep learning policy model outputs recommendations for fine-tuning the target point position or directly retracing and reselecting the trajectory; and finally, the motion command with the highest score is selected as the control output.

[0047] For example, the Proximal Policy Optimization (PPO) algorithm can be used, and the trajectory cost function can be described as:

[0048] in, : The total score of the trajectory, used to select the optimal trajectory for execution; , , , : Weighting coefficient, which controls the weight of each scoring item; : The trajectory angle score towards the target (the closer to the target direction, the higher the score); : The minimum distance score between the trajectory and the obstacle (the farther it is from the safety distance, the higher the score); : Trajectory terminal speed score (encourages faster paths); : The score of the deep policy network (PPO) predicted for the trajectory (indicating the value evaluation of the trajectory in the training policy).

[0049] The expression is as follows:

[0050]

[0051]

[0052]

[0053] in, : Current trajectory angle (the direction of the end of the current trajectory); : Direction angle to the target (the direction angle from the current position to the target point); : The maximum linear speed allowed by AGV; : Minimum obstacle distance on the trajectory; : The maximum evaluation distance is set to 2 meters; : state features corresponding to candidate trajectories; : The parameters are State value network.

[0054] In one possible implementation, the weighting coefficient , , , The initial value of can be set through experience or simulation, such as: , , , In practical applications, the weighting coefficients can be further adjusted through Bayesian optimization. For example, in a dynamic environment, the weights can be dynamically adjusted according to the current task status, such as increasing when approaching the target. , increases when obstacles are dense , if the PPO confidence is high, it increases , thereby improving the dominance of learning strategies.

[0055] The embodiment of the present invention adopts a hierarchical path planning structure. First, the path search algorithm determines the global path, providing a clear target direction and task guidance. Then, in the process of controlling the robot to move along the global path, the dynamic window algorithm is used to optimize the local path, taking into account both target orientation and obstacle avoidance safety, thereby improving the overall responsiveness of the system. Therefore, the embodiment of the present invention realizes the coordinated integration of the global path planning algorithm and the local control optimization strategy, achieving an effective balance between the global rationality of the path and the local dynamic obstacle avoidance capability. In addition, the embodiment of the present invention introduces the strategy network output score obtained by predicting the candidate path through the deep learning strategy model into the trajectory cost function of the dynamic window algorithm. The deep learning strategy model can demonstrate good adaptive ability and long-term decision-making ability in complex dynamic scenes. Therefore, it can learn the optimal path pattern from historical experience, improve the system robustness and task completion efficiency, and make up for the shortcomings of the traditional dynamic window algorithm in the local optimal problem.

[0056] In some embodiments, step S102 includes: determining an initial global path using an A* path planning algorithm; compressing the initial global path using a Floyd path compression algorithm; and smoothing the compressed global path using a cubic B-spline curve.

[0057] The detailed explanation is as follows: (1) Improved A* algorithm Based on the traditional A heuristic path search algorithm, the present invention designs an improved A algorithm that integrates a dynamic weighting mechanism and a minimum heap priority queue to generate a global path from a starting point to an end point in a discrete grid map.

[0058] The core idea of ​​the A* algorithm is based on the cost function , prioritize the nodes to be expanded and select the node with the lowest cost for expansion. The evaluation function is defined as follows:

[0059] in, : The actual cost from the starting point to the current node n.

[0060] : The heuristic estimated cost from the current node n to the target node is calculated using the Euclidean distance:

[0061] : Dynamic weight factor, dynamically adjusted according to the search progress.

[0062] In order to enhance the guidance and efficiency of path search, especially to quickly approach the target area in the early stage of search, the present invention introduces a weighting strategy that is dynamically adjusted based on the progress of the search. Defined as:

[0063] This mechanism realizes the smooth transition of the heuristic function from dominant (initial) to balanced (later), which not only improves the path convergence speed, but also takes into account the stability and rationality of the path. In order to further improve the efficiency of node expansion, the minimum heap structure is used as the priority queue, and the candidate nodes are sorted according to The values ​​are automatically sorted to ensure that the current optimal expansion node is selected each time, reducing the overall search time complexity.

[0064] (2) Floyd path optimization Although the initial path generated by the A* algorithm has connectivity and feasibility from the starting point to the end point, since it searches node by node based on a grid graph, the generated path often has a clear broken line shape and contains a large number of redundant intermediate nodes on the straight path, resulting in an uneven path and frequent turns, which is not conducive to subsequent trajectory smoothing and motion execution.

[0065] To this end, the present invention introduces the Floyd path optimization algorithm to simplify the original A* path sequence. The steps are as follows: Path segment feasibility test: check any three points in the path in sequence 、 、 , if from the starting point Can be directly connected to the destination without any obstacles , then the midpoint They are considered redundant nodes and can be removed from the path; Key point retention mechanism: only necessary turning points and direction change points in the path are retained, and redundant nodes in the path that are consistent with the original direction are eliminated.

[0066] The compressed path not only significantly reduces the number of path nodes and reduces planning complexity, but also has a more regular linear segment structure, which is conducive to subsequent B-spline curve fitting processing.

[0067] Floyd optimization primarily provides a structural foundation for path smoothing. By reducing unnecessary broken line segments, continuous curves (such as cubic B-splines) can generate smooth, continuously differentiable execution paths with fewer control points. This optimization plays a bridging role in the multi-stage path planning process, creating conditions for improving the geometric smoothness of the path while ensuring path feasibility.

[0068] (3) B-spline curve path smoothing After completing Floyd path compression, the resulting path point sequence has significantly reduced redundant nodes, but it still exists in the form of broken line segments. The path exhibits discontinuous first-order derivative changes between nodes, and there are sudden changes at the turning points, which is not conducive to the continuous and smooth execution of the mobile robot.

[0069] In order to further improve the geometric continuity and execution feasibility of the path, this embodiment introduces a cubic B-spline curve to fit the path to achieve path smoothing.

[0070] The B-spline curve takes the path control points as input and generates a curve trajectory that is continuous and second-order differentiable along the entire path. It is defined as follows:

[0071] in, is the i-th path control point; t is a parameterized variable defined within the node interval; is the B-spline basis function, k is the curve order, k=4 in the present invention, and is recursively defined as follows:

[0072]

[0073] Fixed node spacing and control points are set to generate continuous and differentiable path curves to ensure the motion continuity of the AGV execution process.

[0074] During curve construction, by setting equidistant or adaptive node parameters and endpoint constraints, the generated path is ensured to have: Overall continuous and smooth: no sharp corners, satisfying second-order geometric continuity; Physical feasibility: The path is continuous and smooth in space, meeting the requirements for smooth movement of non-omnidirectional wheeled robots; Controllable curvature range: By adjusting the distribution of control points and the spacing between nodes, the path curvature can be limited to avoid trajectory oscillation.

[0075] The resulting smooth path is not only more natural in space, but also better fits the actual kinematic constraints of the AGV, improving the controllability, execution stability, and navigation robustness of the path in a real physical platform.

[0076] In some embodiments, step S103 of predicting the candidate paths using a pre-trained deep learning strategy model to obtain a strategy network output score includes: (1) High-dimensional features of the grid map are extracted through the Graph Neural Network (GNN) to obtain high-dimensional features corresponding to the candidate paths.

[0077] After building a raster map, we can use graph neural networks to model environmental topology, achieving a fusion of environmental structure, semantic categories, and dynamic trends. This involves using graph neural networks to construct a topological map of entities such as obstacles, structures, and traffic areas. Nodes represent recognizable objects, and edges describe their spatial or dynamic relationships. This allows for high-dimensional modeling of environmental structure and dynamic trends, and the extracted features are then referenced by the routing strategy network.

[0078] (2) Based on the high-dimensional features corresponding to the candidate paths and the current state data of the robot, the state space vector of the deep learning policy model is constructed and input into the deep learning policy model. The actor network generates the candidate action distribution, and the critic network outputs the trajectory score value as the policy network output score.

[0079] To enhance the intelligence of path policy decision-making, this implementation introduces the PPO algorithm and establishes a path policy learning subsystem. The system's state space includes the current position, target point, distribution of nearby obstacles, and historical trajectories; the action space includes steering angle and linear velocity; the reward function design prioritizes decreasing distance to the target (positive reward), approaching obstacles (negative penalty), and path smoothness and continuity (auxiliary reward). The network architecture utilizes an actor-critic dual network design, and the training method is online simulation self-learning. Once policy training is complete, it can be used for assisting path node selection, trajectory sampling and scoring, and optimizing local controller feedback, significantly improving path intelligence and generalization capabilities.

[0080] For the PPO algorithm:

[0081] : strategy ratio, : advantage function, : Clipping factor to prevent excessive policy updates.

[0082] The state space and action space are defined as: Status: including current position, target point coordinates, obstacle distribution, path direction, etc.; Motion: including steering angular velocity and linear velocity; The reward function is designed by taking into account the weighted construction of distance to the target, obstacle avoidance distance, and path smoothness. The reward function can be designed as follows:

[0083] : The Euclidean distance from the current AGV to the target point; : The distance to the nearest obstacle on the trajectory (meters), when When the task is terminated, : The change in direction angle between adjacent path points (radians), used to penalize sharp turns; : normalized speed; Parameter settings: , , , , .

[0084] PPO uses the Actor-Critic architecture, generates path interactions with the help of a simulation environment, uses a stable policy update method to overcome the policy jitter problem, and outputs a path policy network to guide the adjustment and planning direction of key nodes in the A* path.

[0085] In some embodiments, calculating the score of each candidate local path using the trajectory cost function in step S103 includes: Obtain the initial weights of the target orientation score, speed maintenance score, obstacle distance score, and policy network output score; For each candidate partial path, the size of each initial weight is dynamically adjusted according to the state data of the robot and the candidate partial path.

[0086] In addition to expanding the dynamic window parameters (i.e., introducing the policy network output score), this embodiment also makes the following improvements to the DWA algorithm in the local control phase: (1) Increase sampling density and improve trajectory accuracy.

[0087] Traditional DWA uses the feasible speed space (line speed and angular velocity ) is sampled to generate the trajectory, and the trajectory accuracy depends on the sampling resolution. In this embodiment, the sampling density is increased in the velocity space, and ( , ) parameter pairs; improve the time-step accuracy of trajectory prediction (reducing the prediction period from 1s to 0.1s); and each velocity pair generates a more refined trajectory point sequence, facilitating obstacle detection and curvature constraint checking. This mechanism significantly improves the resolution and execution accuracy of trajectory candidates in complex environments, effectively enhancing obstacle avoidance sensitivity and navigation smoothness.

[0088] (2) Introduce a dynamic speed adjustment mechanism to deal with sudden obstacles.

[0089] In order to adapt to the temporary changes and uncertain disturbances of dynamic obstacles, a "dynamic speed adjustment mechanism" is introduced to enable the AGV to automatically reduce speed in emergencies and implement an early deceleration and steering strategy: The current speed is determined by the end speed of the current trajectory estimate:

[0090] in, is the current state position, is the position of the next time step, is the trajectory time step.

[0091] When the minimum obstacle distance in the predicted trajectory (safety distance threshold), a speed penalty factor is introduced :

[0092] in It can be defined as an exponentially decreasing function based on the obstacle distance:

[0093] is the distance penalty coefficient (adjusts the penalty rate), and the constant 1 / m controls how quickly the speed decreases when an obstacle approaches. This allows for an adaptive speed strategy of "slower approaching an obstacle" and improves obstacle avoidance robustness.

[0094] In some embodiments, the time dimension can be introduced on the basis of the traditional two-dimensional grid map, and time series modeling methods such as convolutional LSTM can be used to predict the probability of obstacle distribution in several future time steps, realize trajectory prediction and early avoidance, and enhance path robustness and foresight.

[0095] That is: based on the historical sequence of the grid map, a time-series occupancy prediction is performed to obtain a grid map prediction result within a preset time period in the future; if the grid map prediction result shows that the robot's path is occupied by the obstacle area within the preset time period in the future, and the occupancy rate reaches a first preset threshold, then based on the grid map prediction result, the local path optimization is re-executed using the dynamic window algorithm; if the grid map prediction result shows that the robot's path is occupied by the obstacle area within the preset time period in the future, and the occupancy rate reaches a second preset threshold, then based on the grid map prediction result, the global path is determined by the path search algorithm again; wherein the first preset threshold is less than the second preset threshold.

[0096] For example: using convolutional LSTM to analyze historical raster map sequences Make time series occupancy predictions; if the future Occupancy probability of key path points within s 0.7s, trigger local path resampling (local replanning); if 5 consecutive frames are predicted 0.9s, triggering global A* path replanning and subsequent local path replanning.

[0097] Figure 2 This is a detailed implementation flow diagram of the path planning method provided by an embodiment of the present invention. Figure 2 , this embodiment is described: Step 1: Environment modeling and map construction.

[0098] Construct a simulation platform with the size of In order to facilitate path planning and dynamic obstacle avoidance, the entire environment is discretized into The equidistant grid is divided into 400 grid units, which has spatial discreteness and numerical calculation basis.

[0099] Obstacles in the environment adopt a structured design, mainly including: Cross wall structure: simulates partitions or transmission channels in industrial scenes; Two irregular L-shaped obstacles: representing obstacles such as corner shelves and equipment boundaries; The overall obstacle ratio is controlled at approximately 20%, ensuring that path planning is challenging while retaining enough free space for multi-strategy testing.

[0100] The starting coordinates are set to (2,2), the end coordinates are set to (18,18), the passable area is a white grid, and the impassable area (obstacle) is marked in black.

[0101] During the map construction process, visualization and verification mechanisms are integrated: tools are used to render the environment map and display the grid status in real time; a reachability verification module is introduced to verify connectivity from the starting point to the end point, ensuring the existence of at least one path; an API interface is provided for the path search module, DWA controller, and deep strategy network to access the map structure; and a unified and standardized data foundation is provided for subsequent global path planning and local obstacle avoidance control. This map construction method combines structured obstacle layout with random arrangement to realistically simulate the spatial constraints of complex industrial or logistics environments, helping to test the robustness and adaptability of path planning algorithms under multiple obstacle interference.

[0102] Step 2: Implementation of environmental topology modeling based on graph neural network Node Generation: Obstacle areas marked on the static grid map are clustered, with each independent obstacle or structure area considered a node. Node attributes include: center location (coordinates); geometry type (e.g., straight, L-shaped); semantic category (e.g., wall, shelf, boundary); and dynamic label (whether it is covered by or close to a dynamic obstacle).

[0103] Edge construction and feature calculation: If the Euclidean distance between two obstacle nodes is less than a threshold (e.g., 2 meters), an undirected edge is established between them. Edge attributes include: spatial distance; environmental semantic similarity; and dynamic interaction frequency (based on historical motion data statistics).

[0104] Graph neural network reasoning and vector embedding: After building the graph structure, the graph convolutional network (GCN) structure is used to perform propagation calculations on the graph, extract the structural context features of each node, and form an embedding vector.

[0105] The output features serve as the input state vector of the PPO policy network, implementing a perception-policy fusion mechanism. This topology mapping module not only improves path planning's understanding of environmental geometric constraints but also provides structural prior knowledge for subsequent time series prediction and policy scoring.

[0106] Step 3: Determine the initial global path through the A* path planning algorithm.

[0107] Step 4: Compress the initial global path using the Floyd path compression algorithm.

[0108] Step 5: Smooth the compressed global path using a cubic B-spline curve.

[0109] Step 6: Perform local optimization using the DWA algorithm to optimize each candidate trajectory. , extract its state sequence features ; Input to the trained PPO strategy network Actor part and output the strategy score for the trajectory ; In the total score function, it is weighted and evaluated together with the traditional DWA evaluation index; If local traps or redundant paths repeatedly oscillate, the PPO output recommends fine-tuning the target point position, or directly backing off and reselecting the trajectory; Finally, the motion command with the highest score is selected as the control output.

[0110] Step 7: Select the candidate local path with the highest score as the optimal local path.

[0111] Step 8: Use convolutional LSTM to process historical raster map sequences Make time series occupancy predictions; if the future Occupancy probability of key path points within s 0.7s, trigger local path resampling (local replanning); if 5 consecutive frames are predicted 0.9s, triggering global A* path replanning.

[0112] The innovations of this application include: (1) Based on the traditional two-dimensional grid map, the time dimension is introduced, and time series modeling methods such as convolutional LSTM are used to predict the probability of obstacle distribution in several future time steps, so as to achieve trajectory prediction and early avoidance, and enhance the path robustness and foresight.

[0113] (2) The structural features generated by the graph neural network serve as the state embedding vector of the path policy network, participating in the trajectory sampling, scoring and policy updating process, and realizing deep coupling from environmental structure perception to path policy output.

[0114] (3) The PPO reinforcement learning mechanism is introduced to train the policy network to learn the path node selection rules and trajectory quality assessment criteria. The policy network demonstrates good adaptability and long-term decision-making capabilities in complex dynamic scenarios. It can learn the optimal path pattern from historical experience, improve system robustness and task completion efficiency, and make up for the shortcomings of the traditional DWA algorithm in the local optimal problem.

[0115] (4) Adopting a hierarchical path planning structure: The global path planning module generates a global navigation path from the starting point to the end point based on the improved A* algorithm, providing a clear target direction and task guidance; the local path planning module adopts the DWA algorithm, which dynamically samples feasible trajectories based on the current robot state and perception environment on the basis of global path guidance, evaluates and selects the optimal control command in real time, taking into account both target orientation and obstacle avoidance safety, and improving the overall response capability of the system.

[0116] (5) During the path execution process, the system has the ability to actively select information. It can actively obtain information about currently invisible or uncertain environmental areas by adjusting the sensor viewing angle, path offset, and detouring around blocked areas. This mechanism is obtained through reinforcement learning strategy training, which greatly enhances the system's perception of dynamic changes and environmental adaptability.

[0117] (6) This method upgrades the path planning system from the traditional "passive perception-path response" model to a closed-loop mechanism of "structure mapping-dynamic prediction-active interaction-strategy control", achieving a deep integration of global rationality, local flexibility, and environmental adaptability in path planning. This mechanism significantly improves the navigation success rate, trajectory continuity, and system robustness in multi-obstacle dynamic environments.

[0118] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0119] Figure 3 Schematic diagram of a robot provided by an embodiment of the present invention. Figure 3 As shown, the robot 3 of this embodiment includes a processor 30 and a memory 31. Memory 31 stores a computer program 32. When the processor 30 executes computer program 32, the steps of the aforementioned method embodiments are implemented. Alternatively, when the processor 30 executes computer program 32, the functions of the modules / units in the aforementioned device embodiments are implemented.

[0120] For example, the computer program 32 may be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 30 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the robot 3.

[0121] The robot 3 may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3This is only an example of the robot 3 and does not constitute a limitation of the robot 3. The robot 3 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the robot 3 may also include input and output devices, network access devices, buses, etc.

[0122] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0123] Memory 31 can be an internal storage unit of robot 3, such as the robot's hard drive or memory. Memory 31 can also be an external storage device of robot 3, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on robot 3. Furthermore, memory 31 can include both the robot's internal storage unit and an external storage device. Memory 31 is used to store computer program 32 and other programs and data required by robot 3. Memory 31 can also be used to temporarily store data that has been output or is about to be output.

[0124] For the sake of convenience and brevity, the division of the above functional modules / units is only used as an example. In actual applications, the above functions can be assigned to different functional modules / units as needed. The above modules / units can be implemented in the form of hardware, software, or a combination of hardware and software.

[0125] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in the above-mentioned method embodiments.

[0126] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the methods in the above-mentioned method embodiments.

[0127] The term "computer program" includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. Computer-readable media may include any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunications signals, and software distribution media.

[0128] In the above embodiments, the descriptions of each embodiment have their own focus. For parts not described or recorded in detail in one embodiment, please refer to the relevant descriptions of other embodiments. Unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other. The technical features of different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0129] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A path planning method, characterized in that: include: Acquiring environmental information of a target area, and establishing a grid map based on the environmental information, wherein the grid map includes a starting area, an end area, and an obstacle area; Determining a global path using a path search algorithm based on the grid map; In the process of controlling the robot to move along the global path, local path optimization is performed using a dynamic window algorithm; wherein, the dynamic window algorithm calculates the score of each candidate local path and determines the optimal local path through a trajectory cost function, and the trajectory cost function introduces the strategy network output score obtained by predicting the candidate path through a pre-trained deep learning strategy model.

2. The path planning method according to claim 1, wherein: The candidate paths are predicted by the pre-trained deep learning policy model to obtain the policy network output score, including: Extract high-dimensional features from the grid map using a graph neural network to obtain high-dimensional features corresponding to the candidate paths; Based on the high-dimensional features corresponding to the candidate paths and the current state data of the robot, the state space vector of the deep learning strategy model is constructed and input into the deep learning strategy model. The actor network generates the candidate action distribution, and the critic network outputs the trajectory score value as the output score of the strategy network.

3. The path planning method according to claim 1, wherein: The scoring result of the trajectory cost function is a weighted sum of the target orientation score, the speed maintenance score, the obstacle distance score, and the strategy network output score.

4. The path planning method according to claim 1, wherein: When performing local path optimization, the dynamic window algorithm determines the minimum distance between the candidate local path and the obstacle area in the grid map, and determines whether the minimum distance is less than a preset safety distance threshold; If the minimum distance is less than the safety distance threshold, a penalty factor is determined according to the minimum distance, and the minimum distance is positively correlated with the size of the penalty factor; The speed of the robot is dynamically adjusted according to the penalty factor.

5. The path planning method according to claim 3, wherein: The score of each candidate local path is calculated by using the trajectory cost function, including: Obtaining initial weights of the target orientation score, the speed maintenance score, the obstacle distance score, and the policy network output score; For each candidate partial path, the size of each initial weight is dynamically adjusted according to the state data of the robot and the candidate partial path.

6. The path planning method according to any one of claims 1 to 5, characterized in that: The method further comprises: Perform time series occupancy prediction based on the historical sequence of the grid map to obtain grid map prediction results within a preset time period in the future; If the grid map prediction result shows that the robot's path is occupied by an obstacle area within the future preset time period, and the occupancy rate reaches a first preset threshold, then re-execute the local path optimization using the dynamic window algorithm based on the grid map prediction result; If the grid map prediction result shows that the robot's path is occupied by an obstacle area within the future preset time period, and the occupancy rate reaches a second preset threshold, re-executing the determination of the global path by the path search algorithm based on the grid map prediction result; The first preset threshold is smaller than the second preset threshold.

7. The path planning method according to any one of claims 1 to 5, characterized in that: The obstacle area includes: a static obstacle area and a dynamic obstacle area; After establishing the grid map according to the environmental information, the method further includes: The position of the dynamic obstacle area is updated in real time according to the environmental information.

8. The path planning method according to any one of claims 1 to 5, characterized in that: Determining a global path by a path search algorithm based on the grid map includes: Determine the initial global path through the A* path planning algorithm; Compress the initial global path using the Floyd path compression algorithm; The compressed global path is smoothed using a cubic B-spline curve.

9. The path planning method according to claim 8, wherein: The heuristic function of the A* path planning algorithm is the weighted sum of the actual cost from the starting point to the current node and the heuristic estimated cost from the current node to the target node; and the weight corresponding to the heuristic estimated cost from the current node to the target node is dynamically adjusted according to the search progress.

10. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images