A vehicle dynamic path planning method and system based on improved RRT

By combining the improved RRT algorithm with Digestella and repeat sequence optimization, combined with dynamic window method and deep reinforcement learning model, path planning is optimized, and the problem of low path planning efficiency in complex environments is solved, and the vehicle is realized Fast and effective path planning in complex environments.

CN115373384BActive Publication Date: 2024-11-29ANHUI NORMAL UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210894584.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-28
Publication Date
2024-11-29
Estimated Expiration
2042-07-28

AI Technical Summary

Technical Problem

When traditional RRT algorithms plan paths in complex dynamic environments, they have large amounts of calculations and many redundant leaf nodes, resulting in low time efficiency and high storage consumption, which cannot meet the vehicle's real-time obstacle avoidance requirements.

Method used

The improved RRT algorithm is used, combined with the Digestella algorithm and repeat sequence optimization, and the initial path is generated. When there are dynamic obstacles, the dynamic window method and the deep reinforcement learning model are used to optimize the path to generate the final path.

Benefits of technology

It reduces the time consumption of path planning, improves the efficiency of path planning, and the generated path is smoother and smoother, meeting the vehicle's requirements for rapid and effective endpoint arrival.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115373384B_ABST
    Figure CN115373384B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a vehicle dynamic path planning method and system based on improved RRT, which belongs to the technical field of path planning. The vehicle dynamic path planning method based on improved RRT includes: establishing a grid map model, determining the starting point and the end point; using the improved RRT algorithm to obtain the initial path, wherein the initial path is configured as a node expansion strategy based on adjacency relationship and integrated with the traditional RRT algorithm to search in the free space of the grid map model; obtaining the first path after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm; when there are dynamic obstacles, combining the dynamic window method and the deep reinforcement learning model to perform path optimization to update the first path to obtain the second path as the final path. The present invention can reduce sampling randomness, improve path planning efficiency, and is applied to vehicle dynamic path planning in complex environments with dynamic obstacles.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of path planning, and in particular to a vehicle dynamic path planning method and system based on improved RRT. Background Art

[0002] With the rapid development of my country's economy and the deepening of urbanization, the contradiction between supply and demand of urban transportation has become increasingly prominent. More and more people own private cars. Urban congestion and changeable road conditions have brought inconvenience to people's lives, but the vehicle dynamic path planning system can bring great convenience to people's travel. The vehicle dynamic path planning system collects the surrounding environmental information through sensors and processes it to guide the vehicle's driving. After obtaining the environmental information, an environmental model is established. Through computer calculation and processing, the feedback information is transmitted to the control system, so that the vehicle can effectively avoid some temporary dynamic obstacles and arrive at the destination smoothly. The core technology of the vehicle dynamic path planning system is the path planning algorithm. Its main idea is to combine the current environment, generate raster map data, and successfully plan a collision-free feasible path in the free space from the current vehicle position to the destination.

[0003] Traditional path planning algorithms include artificial potential field method based on structural potential field, genetic algorithm, ant colony algorithm and A* based on graph search, etc. However, the artificial potential field method based on structural potential field will increase the amount of calculation in the environment with irregular obstacles, and it is easy to have force balance in front of obstacles, resulting in stagnation or oscillation within the range. Genetic algorithm and ant colony algorithm have high computational cost and are difficult to apply in actual path planning. The spatial growth of A* based on graph search is exponential. The above algorithms are not suitable for path planning in complex dynamic environments. The advantages of Rapid-exploration Random Tree (RRT) are its simple structure, strong search capability, easy to combine with other algorithms, can avoid modeling the state space, and solve the path planning problems of high-dimensional space and complex constraints.

[0004] The traditional RRT algorithm uses an initial node as the root node, adds leaf nodes through random sampling, and generates a random expansion tree. Usually, because of the need to solve the complexity of the path space, the random expansion tree will generate more redundant leaf nodes, which not only reduces the time efficiency of the algorithm, but also increases the storage consumption of the algorithm and the length of the shortest path generated. The planned path cannot meet the real-time obstacle avoidance requirements of the vehicle, and thus cannot meet the vehicle's quick and effective arrival at the destination. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a vehicle dynamic path planning method and system based on improved RRT, which can reduce sampling randomness and improve path planning efficiency, and is applied to vehicle dynamic path planning in complex environments with dynamic obstacles.

[0006] In order to achieve the above object, an embodiment of the present invention provides a vehicle dynamic path planning method based on improved RRT, and the vehicle dynamic path planning method based on improved RRT includes:

[0007] Establish a raster map model and determine the starting and ending points;

[0008] An initial path is obtained by using an improved RRT algorithm, wherein the initial path is configured as a node expansion strategy based on an adjacency relationship and integrated with a traditional RRT algorithm to search in the free space of the grid map model;

[0009] Obtaining a first path after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm; and

[0010] When there are dynamic obstacles, the dynamic window method and the deep reinforcement learning model are combined to perform path optimization to update the first path to obtain the second path as the final path.

[0011] Preferably, the step of establishing a grid map model and determining a starting point and an end point comprises:

[0012] Define a fast exploration random tree T based on adjacency relationship, initialize a tree T with only node D and a queue Q, and add the target node D to the queue Q;

[0013] Furthermore, the method of obtaining the initial path by using the improved RRT algorithm includes:

[0014] Define Adjacent(v) as the set of nodes that are adjacent to node v and can be directly accessed from node v. Node c∈Adjacent(v) is the first node v. t Out of the team, Adjacent (v t ) into the queue. If the node v that is adjacent to the head node and can be directly accessed has not been explored, the shortest path length from the adjacent nodes of node v to the target node D is calculated according to formula (1). The node with the smallest path length is selected to connect node v as the parent node of node v in T. The path length between node v and the parent node is recorded in U according to formula (2). v In, until the queue Q becomes empty;

[0015]

[0016] U v ←Uv.parent +distance(v.parent,v) (2)

[0017] In formula (1), the shortest path length from node v to target node D is represented by U v , define distance(v i ,v j )≥0 for any two nodes v i With v j The Euclidean distance length between nodes, the parent node of node v in the improved RRT spanning tree is represented as v.parent;

[0018] Preferably, the first path obtained after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm as the final path includes:

[0019] Methods for optimizing paths based on the Dijkstra algorithm include:

[0020] In the initial path sequence P init If any two nodes v i and v j If there is no obstacle node blocking the connection between them on the map M, then in the initial path sequence P init Add edge <v i ,v j >

[0021] Initialize the node sets V and L to be empty, and set V init The node v is added to the set V and the shortest distance dist(v) and prev(v) between the node v and S are calculated. Using the Dijkstra algorithm, the initial path sequence P init Search for the shortest path P from the starting node S to the target node D;

[0022] Delete the node u with the smallest dist(v) from V and add it to L. Update the shortest distance from the starting node S to v and save u as the previous node of node v in prev(v) until the queue Q becomes empty.

[0023] Initialize the path sequence P, starting from the target node D, add the previous node of node v in the ordered shortest distance sequence from the starting node S to node v, and obtain the Dijkstra path sequence P;

[0024] Methods for optimizing paths based on repeated sequence optimization algorithms include:

[0025] Initialize the path sequence P opt , where there is only the starting node S in the node set, the edge set is empty, and the values ​​of Cur_S and Cur_D are initialized;

[0026] Find the node v Cur_S and v Cur_D The maximum value of Cur_D when there is no obstacle between the connection nodes; if the node v Cur_D If it is not the target node D, then perform the following steps: If Cur_D-1 and Cur_S+1 are equal, then go to E opt Add edge <v Cur_S ,v Cur_S+1 >, otherwise add the edge <v Cur_S ,v Cur_D-1 >, update the values ​​of Cur_S and Cur_D and set the node v Cur_S Join V opt Repeat the cycle until the node v Cur_D is the target node D;

[0027] In the optimized path sequence P opt V opt Add the target node D in E opt Add edge <v Cur_S ,D>, the path sequence optimization work is completed.

[0028] Preferably, the step of using the path obtained after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm as the final path is configured to be performed when there are no dynamic obstacles.

[0029] Preferably, the combining of the dynamic window method and the deep reinforcement learning model to perform path optimization to update the first path to obtain the second path as the final path includes:

[0030] Step 1: construct a model of the dynamic window method based on deep reinforcement learning and its optimization target;

[0031] Step 2: Initialize all parameter values ​​of the model based on the dynamic window method of deep reinforcement learning and randomly generate a data set;

[0032] Step 3, constructing a training process of the model, inputting the randomly generated data set into the model, and calculating the reward value of each round of training results;

[0033] Step 4: Optimize the model using a dynamic window method based on deep reinforcement learning according to the loss value;

[0034] Step 5: Set the maximum number of training rounds, repeat steps 3 and 4 to obtain a trained model, use the trained model to perform dynamic obstacle avoidance, and update the optimal path.

[0035] Preferably, the training process of constructing the model, inputting the randomly generated data set into the model, and calculating the reward value of each round of training results includes:

[0036] The randomly generated data set is input into the vehicle's motion environment to obtain information about the surrounding environment and the current state, and then the data set is processed using a convolutional neural network, and the agent interacts with the environment in the current state;

[0037] Explore the environment, select actions by strategy, add the weight of the evaluation function of the dynamic window method corresponding to the action to the initial weight, select the optimal speed corresponding to the maximum evaluation function in the speed window, and move to the state of the next control cycle. A reward will be generated during the interaction process, and the data will be stored in the experience storage pool.

[0038] Preferably, the optimizing the model by using a dynamic window method based on deep reinforcement learning according to the loss value includes:

[0039] When the number of samples in the experience storage pool is greater than n, a group of samples are uniformly and randomly selected from the experience storage pool;

[0040] The extracted set of samples is sent to the optimization objective function to update the network parameters.

[0041] In addition, the present invention also provides a vehicle dynamic path planning system based on improved RRT, and the vehicle dynamic path planning system based on improved RRT includes:

[0042] A model building unit is used to build a grid map model and determine the starting point and the end point;

[0043] An initial path acquisition unit, used for acquiring an initial path by using an improved RRT algorithm, wherein the initial path is configured as a node expansion strategy based on an adjacency relationship and integrated with a traditional RRT algorithm to search in the free space of the grid map model;

[0044] A first path obtaining unit, configured to obtain a first path after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm; and

[0045] A path updating unit is used to optimize the path by combining a dynamic window method and a deep reinforcement learning model to update the first path to obtain a second path as a final path when there are dynamic obstacles.

[0046] In addition, the present invention also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned vehicle dynamic path planning method based on improved RRT.

[0047] In addition, the present invention also provides a processor, which is used to run a program, wherein the vehicle dynamic path planning method based on improved RRT is executed when the program is running.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. The node expansion strategy based on adjacency relationship reduces the randomness of RRT algorithm sampling and improves the search efficiency of random tree, thus shortening the planning time and improving the efficiency of path planning in vehicle path planning;

[0050] 2. The Dijkstra path sequence search algorithm and repeated sequence optimization algorithm are used to prune the planned path, which reduces the time consumption, reduces the length of the planned path, and makes the planned path more stable and smooth.

[0051] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0053] Figure 1 It is a schematic flow chart of a specific embodiment of a vehicle dynamic path planning method based on improved RRT of the present invention;

[0054] Figure 2 It is a schematic diagram of the result of optimizing the initial path in a complex environment by the vehicle dynamic path planning method based on the improved RRT of the present invention;

[0055] Figure 3 It is a schematic flow chart of another embodiment of the vehicle dynamic path planning method based on improved RRT of the present invention;

[0056] Figure 4 A schematic diagram of dynamic obstacle trajectories in a complex environment using the vehicle dynamic path planning method based on improved RRT of the present invention;

[0057] Figure 5 This is a schematic diagram of the result of finding the optimal path in a complex environment using the vehicle dynamic path planning method based on the improved RRT of the present invention;

[0058] Figure 6 It is a structural schematic diagram of a specific embodiment of the vehicle dynamic path planning system based on improved RRT of the present invention. DETAILED DESCRIPTION

[0059] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0060] The present invention discloses a vehicle dynamic path planning method based on improved RRT, such as Figure 1 As shown,

[0061] S101, establishing a grid map model and determining the starting point and the end point;

[0062] S102, using an improved RRT algorithm to obtain an initial path, wherein the initial path is configured as a node expansion strategy based on an adjacency relationship and integrated with a traditional RRT algorithm to search in the free space of the grid map model; and

[0063] S103, a first path obtained after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm;

[0064] S104: When there are dynamic obstacles, a dynamic window method and a deep reinforcement learning model are combined to perform path optimization to update the first path to obtain a second path as a final path.

[0065] Among them, in step S101,

[0066] The said establishing a grid map model and determining the starting point and the end point comprises:

[0067] Define a fast exploration random tree T based on adjacency relationship, initialize a tree T with only node D and a queue Q, and add the target node D to the queue Q;

[0068] Furthermore, the method of obtaining the initial path by using the improved RRT algorithm includes:

[0069] Define Adjacent(v) as the set of nodes that are adjacent to node v and can be directly accessed from node v. Node c∈Adjacent(v) is the first node v. t Out of the team, Adjacent (v t ) into the queue. If the node v that is adjacent to the head node and can be directly accessed has not been explored, the shortest path length from the adjacent nodes of node v to the target node D is calculated according to formula (1). The node with the smallest path length is selected to connect node v as the parent node of node v in T. The path length between node v and the parent node is recorded in U according to formula (2). v In, until the queue Q becomes empty;

[0070]

[0071] U v ←U v.parent +distance(v.parent,v) (2)

[0072] In formula (1), the shortest path length from node v to target node D is represented by U v , define distance(v i ,v j )≥0 for any two nodes v i With v j The Euclidean distance length between nodes, the parent node of node v in the improved RRT spanning tree is represented as v.parent;

[0073] Preferably, if Figure 2 As shown, the first path obtained after optimization based on the Dijkstra algorithm (Dijkstra path sequence search algorithm) and the repeated sequence optimization algorithm as the final path includes:

[0074] The Dijkstra path sequence search algorithm and the repeated sequence optimization algorithm are used to perform pruning optimization and smoothing on the initial path respectively, and the optimized path (the trajectory is represented by a dotted line) from the starting point (represented by a triangle symbol) to the end point (represented by a dot symbol) is obtained. The method of optimizing the path based on the Dijkstra algorithm includes:

[0075] In the initial path sequence P init If any two nodes v i and v j If there is no obstacle node blocking the connection between them on the map M, then in the initial path sequence P init Add edge <v i ,v j >

[0076] Initialize the node sets V and L to be empty, and set V init The node v is added to the set V and the shortest distance dist(v) and prev(v) between the node v and S are calculated. Using the Dijkstra algorithm, the initial path sequence P init Search for the shortest path P from the starting node S to the target node D;

[0077] Delete the node u with the smallest dist(v) from V and add it to L. Update the shortest distance from the starting node S to v and save u as the previous node of node v in prev(v) until the queue Q becomes empty.

[0078] Initialize the path sequence P, starting from the target node D, add the previous node of node v in the ordered shortest distance sequence from the starting node S to node v, and obtain the Dijkstra path sequence P;

[0079] Methods for optimizing paths based on repeated sequence optimization algorithms include:

[0080] Initialize the path sequence P opt, where there is only the starting node S in the node set, the edge set is empty, and the values ​​of Cur_S and Cur_D are initialized;

[0081] Find the node v Cur_S and v Cur_D The maximum value of Cur_D when there is no obstacle between the connection nodes; if the node v Cur_D If it is not the target node D, then perform the following steps: If Cur_D-1 and Cur_S+1 are equal, then go to E opt Add edge <v Cur_S ,v Cur_S+1 >, otherwise add the edge <v Cur_S ,v Cur_D-1 >, update the values ​​of Cur_S and Cur_D and set the node v Cur_S Join V opt Repeat the cycle until the node v Cur_D is the target node D;

[0082] In the optimized path sequence P opt V opt Add the target node D in E opt Add edge <v Cur_S ,D>, the path sequence optimization work is completed.

[0083] Preferably, the step of using the path obtained after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm as the final path is configured to be performed when there are no dynamic obstacles.

[0084] Preferably, if Figure 3 As shown, the vehicle dynamic path planning method based on improved RRT may also include:

[0085] When there are dynamic obstacles, the dynamic window method and the deep reinforcement learning model are combined to perform path optimization to update the first path to obtain the second path as the final path.

[0086] Specifically, the steps may include:

[0087] Step 1: first, use the vehicle's own laser radar sensor or depth camera sensor to collect environmental information around the vehicle, and use it to establish a grid map, mark each grid in the grid map as an obstacle space or a free space, and select the starting point and the end point in the marked grid map;

[0088] Step 2, obtain the initial path by improving the RRT algorithm;

[0089] Step 3, obtaining a first path after optimization based on Dijkstra algorithm (Dijkstra path sequence search algorithm) and repeated sequence optimization algorithm;

[0090] Step 4, taking the path points on the optimized path as virtual sub-target points of the improved dynamic window method;

[0091] Step 5: Determine the current virtual sub-target point.

[0092] Step 6, specifically, Figure 4 As shown, the vehicle realizes real-time detection of the surrounding environment by improving the dynamic window method. When an unknown obstacle (indicated by a five-pointed star symbol) and a dynamic obstacle (indicated by a green symbol and a red symbol) are found, and the dynamic obstacle moves in a random direction (the trajectory is represented by a solid line), in order to avoid collision with it, the vehicle performs dynamic obstacle avoidance by improving the dynamic window method and updates the optimal path. The path optimization combining the dynamic window method and the deep reinforcement learning model to update the first path to obtain the second path as the final path includes:

[0093] Step 6.1, construct a model of the dynamic window method based on deep reinforcement learning and its optimization target;

[0094] Step 6.2, initialize all parameter values ​​of the model based on the dynamic window method of deep reinforcement learning, and randomly generate a data set;

[0095] Step 6.3, constructing a training process of the model, inputting the randomly generated data set into the model, and calculating the reward value of each round of training results;

[0096] Step 6.4, optimize the model using the dynamic window method based on deep reinforcement learning according to the loss value;

[0097] Step 6.5, set the maximum number of training rounds, repeat steps 6.3-6.4 to obtain the trained model, use the trained model to perform dynamic obstacle avoidance, and update the optimal path.

[0098] Preferably, the training process of constructing the model, inputting the randomly generated data set into the model, and calculating the reward value of each round of training results includes:

[0099] 6.3.1, input the randomly generated data set into the vehicle's motion environment to obtain information about the surrounding environment and the current state, and then use a convolutional neural network to process the data set, and the intelligent agent interacts with the environment in the current state;

[0100] 6.3.2, explore the environment, select actions by strategy, add the weight of the evaluation function of the dynamic window method corresponding to the action to the initial weight, select the optimal speed corresponding to the maximum evaluation function in the speed window, and move to the state of the next control cycle. A reward will be generated during the interaction process, and the data will be stored in the experience storage pool.

[0101] Preferably, the optimizing the model by using a dynamic window method based on deep reinforcement learning according to the loss value includes:

[0102] Step 6.4.1, when the number of samples in the experience storage pool is greater than n, a group of samples is uniformly and randomly selected from the experience storage pool;

[0103] In step 6.4.2, the extracted set of samples is sent to the optimization objective function to update the network parameters.

[0104] Step 7, such as Figure 5 As shown, after moving to the virtual sub-target point (the trajectory is represented by a blue solid line), it is determined whether the current virtual sub-target point is the end point. If it is, the path planning is considered to be completed. Otherwise, the next virtual sub-target point is determined and the process jumps to step 5.

[0105] In addition, if Figure 6 As shown, the present invention also provides a vehicle dynamic path planning system based on improved RRT, and the vehicle dynamic path planning system based on improved RRT includes:

[0106] A model building unit is used to build a grid map model and determine the starting point and the end point;

[0107] an initial path acquisition unit, configured to acquire an initial path by using an improved RRT algorithm, wherein the initial path is configured as a node expansion strategy based on an adjacency relationship and integrated with a traditional RRT algorithm to search in the free space of the grid map model; and

[0108] A first path obtaining unit, used for obtaining a first path after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm;

[0109] A path updating unit is used to optimize the path by combining a dynamic window method and a deep reinforcement learning model to update the first path to obtain a second path as a final path when there are dynamic obstacles.

[0110] An embodiment of the present invention provides a storage medium on which a program is stored. When the program is executed by a processor, the vehicle dynamic path planning method based on improved RRT is implemented.

[0111] An embodiment of the present invention provides a processor, which is used to run a program, wherein the vehicle dynamic path planning method based on improved RRT is executed when the program is running.

[0112] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0113] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0114] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0116] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0117] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0118] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0119] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0120] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0121] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A vehicle dynamic path planning method based on improved RRT, characterized in that: The vehicle dynamic path planning method based on improved RRT includes: Establish a raster map model and determine the starting and ending points; An initial path is obtained by using an improved RRT algorithm, wherein the initial path is configured as a node expansion strategy based on an adjacency relationship and integrated with a traditional RRT algorithm to search in the free space of the grid map model; Obtaining a first path after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm; and When there are dynamic obstacles, the path optimization is performed by combining the dynamic window method and the deep reinforcement learning model to update the first path to obtain the second path as the final path; The said establishing a grid map model and determining the starting point and the end point comprises: Define a fast exploration random tree T based on adjacency relationship, initialize a tree T with only node D and a queue Q, and add the target node D to the queue Q; Furthermore, the method of obtaining the initial path by using the improved RRT algorithm includes: Define Adjacent(v) as the set of nodes that are adjacent to node v and can be directly accessed from node v. Node c∈Adjacent(v) is the first node v. t Out of the team, Adjacent (v t ) into the queue. If the node v that is adjacent to the head node and can be directly accessed has not been explored, the shortest path length from the adjacent nodes of node v to the target node D is calculated according to formula (1). The node with the smallest path length is selected to connect node v as the parent node of node v in T. The path length between node v and the parent node is recorded in U according to formula (2). v In, until the queue Q becomes empty; U v ←U v.parent +distance(v.parent,v)(2) In formula (1), the shortest path length from node v to target node D is represented by U v , define distance(v i ,v j )≥0 for any two nodes v i With v j The Euclidean distance length between nodes, the parent node of node v in the improved RRT spanning tree is represented as v.parent; The first path obtained after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm as the final path includes: Methods for optimizing paths based on the Dijkstra algorithm include: In the initial path sequence P init If any two nodes v i and v j If there is no obstacle node blocking the connection between them on the map M, then in the initial path sequence P init Add edge <v i ,v j > Initialize the node sets V and L to be empty, and set V init The node v is added to the set V and the shortest distance dist(v) and prev(v) between the node v and S are calculated. Using the Dijkstra algorithm, the initial path sequence P init Search for the shortest path P from the starting node S to the target node D; Delete the node u with the smallest dist(v) from V and add it to L. Update the shortest distance from the starting node S to v and save u as the previous node of node v in prev(v) until the queue Q becomes empty. Initialize the path sequence P, starting from the target node D, add the previous node of node v in the ordered shortest distance sequence from the starting node S to node v, and obtain the Dijkstra path sequence P; Methods for optimizing paths based on repeated sequence optimization algorithms include: Initialize the path sequence P opt , where there is only the starting node S in the node set, the edge set is empty, and the values ​​of Cur_S and Cur_D are initialized; Find the node v Cur_S and v Cur_D The maximum value of Cur_D when there is no obstacle between the connection nodes; if the node v Cur_D If it is not the target node D, then perform the following steps: If Cur_D-1 and Cur_S+1 are equal, then go to E opt Add edge <v Cur_S ,v Cur_S+1 >, otherwise add the edge <v Cur_S ,v Cur_D-1 >, update the values ​​of Cur_S and Cur_D and set the node v Cur_S Join V opt Repeat the cycle until the node v Cur_D is the target node D; In the optimized path sequence P opt V opt Add the target node D in E opt Add edge <v Cur_S ,D>, the path sequence optimization work is completed.

2. The vehicle dynamic path planning method based on improved RRT according to claim 1 is characterized in that: The step of taking the path obtained after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm as the final path is configured to be performed when there are no dynamic obstacles.

3. The vehicle dynamic path planning method based on improved RRT according to claim 2 is characterized in that: The combining of the dynamic window method and the deep reinforcement learning model to perform path optimization to update the first path to obtain the second path as the final path includes: Step 1: construct a model of the dynamic window method based on deep reinforcement learning and its optimization target; Step 2: Initialize all parameter values ​​of the model based on the dynamic window method of deep reinforcement learning and randomly generate a data set; Step 3, constructing a training process of the model, inputting the randomly generated data set into the model, and calculating the reward value of each round of training results; Step 4: Optimize the model using a dynamic window method based on deep reinforcement learning according to the loss value; Step 5: Set the maximum number of training rounds, repeat steps 3 and 4 to obtain a trained model, use the trained model to perform dynamic obstacle avoidance, and update the optimal path.

4. The vehicle dynamic path planning method based on improved RRT according to claim 3 is characterized in that: The training process of constructing the model, inputting the randomly generated data set into the model, and calculating the reward value of each round of training results includes: The randomly generated data set is input into the vehicle's motion environment to obtain information about the surrounding environment and the current state, and then the data set is processed using a convolutional neural network, and the agent interacts with the environment in the current state; Explore the environment, select actions by strategy, add the weight of the evaluation function of the dynamic window method corresponding to the action to the initial weight, select the optimal speed corresponding to the maximum evaluation function in the speed window, and move to the state of the next control cycle. A reward will be generated during the interaction process, and the data will be stored in the experience storage pool.

5. The vehicle dynamic path planning method based on improved RRT according to claim 4 is characterized in that: The optimization of the model by using the dynamic window method based on deep reinforcement learning according to the loss value includes: When the number of samples in the experience storage pool is greater than n, a group of samples are uniformly and randomly selected from the experience storage pool; The extracted set of samples is sent to the optimization objective function to update the network parameters.

6. A vehicle dynamic path planning system based on improved RRT, characterized in that: The vehicle dynamic path planning system based on the improved RRT uses the method described in any one of claims 1 to 5, and the vehicle dynamic path planning system based on the improved RRT includes: A model building unit is used to build a grid map model and determine the starting point and the end point; an initial path acquisition unit, configured to acquire an initial path by using an improved RRT algorithm, wherein the initial path is configured as a node expansion strategy based on an adjacency relationship and integrated with a traditional RRT algorithm to search in the free space of the grid map model; and A first path obtaining unit, used for obtaining a first path after optimization based on the Dijkstra algorithm and the repeated sequence optimization algorithm; A path updating unit is used to optimize the path by combining a dynamic window method and a deep reinforcement learning model to update the first path to obtain a second path as a final path when there are dynamic obstacles.

7. A machine-readable storage medium having instructions stored thereon, characterized in that: The instruction is used to enable the machine to execute the vehicle dynamic path planning method based on improved RRT as described in any one of claims 1-5.

8. A processor for running a program, wherein: When the program is running, the vehicle dynamic path planning method based on improved RRT described in any one of claims 1 to 5 is executed.

Citation Information

Patent Citations

  • Vehicle path planning method based on improved bidirectional informed-RRT*

    CN113219998A

  • Navigation obstacle avoidance control method, system and model combining path planning and reinforcement learning

    CN114564016A

  • Systems and methods for adaptive path planning

    WO2021062891A1