Path planning method and device, equipment, storage medium and program product

Abstract

The invention discloses a path planning method and device, equipment, a storage medium and a program product. Obtaining a starting pose, a target pose, a preset exploration direction set and a grid map, determining the position of an obstacle from the grid map, inputting the current pose, the exploration direction set and the position of the obstacle into a pre-trained exploration model, and obtaining an evaluation result of each exploration direction, the method comprises the following steps: evaluating a target pose, selecting a plurality of candidate exploration directions of the current pose according to an evaluation result to determine a plurality of candidate next poses of the current pose, determining a residual distance of the next pose according to the candidate next poses and the target pose, and then selecting the next pose and adding the next pose to a planned path. And when the next pose is out of the preset distance range of the target pose, updating the current pose to the next pose, and when the next pose is in the preset distance range of the target pose, connecting the next pose and the target pose through an RS curve, and ending path planning. The path planning time is shortened, and the transportation efficiency of the unmanned mine car is improved.

Description

Technical Field

[0001] The present application relates to the technical field of reinforcement learning and path planning, and in particular to a path planning method, device, equipment, storage medium and program product. Background Art

[0002] With the development of unmanned driving technology, unmanned vehicles are widely used in mining scenarios to solve the problems of difficulty in recruiting workers, frequent accidents and high costs encountered in the transportation process of mining scenarios. The core link of unmanned mining vehicles is path planning. The role of path planning technology is to plan a safe, efficient and kinematically feasible path from the starting point to the end point based on environmental information.

[0003] Existing technologies include many path planning algorithms, such as the A* algorithm in graph search algorithms. However, since the A* algorithm needs to traverse a large number of nodes to find the optimal path, as the complexity of the environment increases, the number of nodes will also increase dramatically. This process will become very time-consuming when the number of nodes is large, ultimately resulting in a long planning time.

[0004] However, due to the complex and changeable environment of mining scenes, the path planning time is too long, which will cause the actual planned path to be out of touch with the actual mining scene, and it will be unable to effectively respond to changes in the mining environment, thereby reducing the transportation efficiency of unmanned mining vehicles. Summary of the invention

[0005] The present application provides a path planning method, device, equipment, storage medium and program product to solve the problem that the existing path planning algorithm takes too long, resulting in the actual planned path being out of touch with the actual mining scene, thereby reducing the transportation efficiency of unmanned mining vehicles.

[0006] In a first aspect, the present application provides a path planning method, comprising:

[0007] Obtaining the starting posture, target posture, preset exploration direction set and grid map of the driving environment corresponding to the path planning, and taking the starting posture as the current posture;

[0008] Determine the obstacle position closest to the current posture from the grid map, input the current posture, the exploration direction set and the obstacle position into a pre-trained exploration model, obtain an evaluation result of each exploration direction in the exploration direction set, and screen out multiple candidate exploration directions of the current posture from the exploration direction set according to the evaluation result;

[0009] Determine multiple candidate next poses of the current pose according to the current pose and the multiple candidate exploration directions, and determine the remaining distance of each candidate next pose according to each candidate next pose and the target pose, wherein the remaining distance is the distance between the candidate next pose and the target pose;

[0010] Selecting a next pose from a plurality of candidate next poses according to each of the remaining distances, and adding the next pose to the planned path;

[0011] When the next posture is outside the preset distance range of the target posture, updating the current posture to the next posture, and returning to the step of determining the obstacle position closest to the current posture from the grid map;

[0012] When the next posture is within a preset distance range of the target posture, the next posture and the target posture are connected through an RS curve, and when the connection is successful, the path planning is ended.

[0013] In a possible implementation, adding the next posture to the planned path includes:

[0014] Extracting a next position and a next posture from the next position, and extracting a first distance between a grid occupied by the vehicle and the axle when the vehicle is located at the next position in the next posture from a pre-generated collision detection table;

[0015] Determine, according to the first distance, a first grid range occupied by the vehicle when it is located at the next position in the next posture;

[0016] Extracting obstacle information within the first grid range from the grid map;

[0017] If there is no obstacle information within the first grid range, adding the next posture to the planned path;

[0018] The step of connecting the next posture and the target posture through an RS curve includes:

[0019] Dividing the RS curve into a plurality of continuous posture points;

[0020] Extracting a first position and a first posture from each of the posture points, and extracting a second distance between a grid occupied by the vehicle and the axle when the vehicle is located at the first position in the first posture from a pre-generated collision detection table;

[0021] Determine, according to the second distance, a second grid range occupied by the vehicle when it is located at the first position in the first posture;

[0022] Extracting obstacle information within the second grid range from the grid map;

[0023] If there is no obstacle information within the second grid range of each of the RS curves, the next posture and the target posture are connected through the RS curve.

[0024] In a possible implementation, the pre-training process of the exploration model includes:

[0025] Obtaining a training start posture, and using the training start posture as a training current posture;

[0026] Determine the obstacle position closest to the training current posture from the grid map as the training obstacle position, so as to input the training current posture, the exploration direction set and the training obstacle position into the exploration model to obtain the target exploration direction in the exploration direction set;

[0027] According to the training current posture and the target exploration direction, determine the next posture of the training current posture as the training next posture, and calculate the reward value of the training next posture according to the training next posture;

[0028] At the end of path planning, the total reward value is determined based on the reward values ​​of each posture on the planned path;

[0029] If the total reward value does not meet the preset convergence condition, the exploration model is adjusted, and the step of returning to the step of determining the obstacle position closest to the current training posture from the grid map as the training obstacle position is performed to perform the next round of iteration;

[0030] If the total reward value meets the preset convergence condition, the training ends.

[0031] In a possible implementation, the calculating the reward value of the next posture according to the training next posture includes:

[0032] Determining the obstacle position closest to the training next posture from the grid map according to the training next posture;

[0033] Determining an obstacle collision reward for the training next pose according to the obstacle position closest to the training next pose and the training next pose;

[0034] Determining a remaining distance reward for the training next posture according to the training next posture and the target posture;

[0035] Calculating a reward value of the training next posture according to an obstacle collision reward of the training next posture and a remaining distance reward of the training next posture;

[0036] The calculation formula for the reward value of the training next posture is as follows:

[0037]

[0038] Among them, r obs To train the obstacle collision reward for the next pose, r goal To train the remaining distance reward for the next posture, XC t+1 and YC t+1 is the 2D coordinate of the position included in the training next pose, XO t+1 and YO t+1 is the two-dimensional coordinate of the obstacle position closest to the training next pose, D is the preset obstacle detection radius, h reward is a coefficient greater than 0, h max 、h min are the maximum distance and minimum distance between each position in the grid map and the target position, respectively, t+1 To train the distance between the next pose and the target pose;

[0039] The total reward value is determined according to the reward value of each posture on the planned path, including:

[0040] Obtain the preset planning success reward, the preset planning failure reward, and the preset single exploration reward. When the path planning is successful, the planning success reward is greater than 0, and the planning failure reward is 0. When the path planning fails, the planning success reward is 0, the planning failure reward is less than 0, and the preset single exploration reward is less than 0.

[0041] Determine a total reward value according to the planning success reward, the planning failure reward, the single exploration reward and the reward value of each of the postures;

[0042] The calculation formula of the total reward value is as follows:

[0043]

[0044] Among them, R is the total reward value, r terminated For the preset planning success reward, r truncated is the preset reward for planning failure, r t+1 is the reward value of the t+1th posture, N is the number of postures included in the planned path during training, and t is an integer greater than or equal to 1 and less than or equal to N-1.

[0045] In a possible implementation manner, after the path planning is finished, the method further includes:

[0046] Extracting one or more discrete pose pairs from the planned path;

[0047] For each discrete pose pair, two discrete poses in the discrete pose pair are used as two endpoints to fit an arc between the two endpoints;

[0048] A posture is inserted on the arc according to a preset distance, and the posture is added to the planned path, the inserted posture is located between two discrete postures of the discrete posture pair in sequence, and the posture in the inserted posture is consistent with the tangent direction of the arc.

[0049] In a possible implementation, the method further includes:

[0050] Obtaining the angle change between adjacent postures on the planned path;

[0051] If at least one of the angle changes on the planned path is greater than or equal to a preset angle, the planned path is adjusted, and the preset angle is determined according to the maximum turning direction of the vehicle.

[0052] In a possible implementation, the method further includes:

[0053] Determine the driving direction according to any two adjacent positions on the planned path;

[0054] Determining the number of switches to the opposite direction on the planned path according to the plurality of driving directions on the planned path;

[0055] If the number of opposite direction switches is greater than or equal to a preset number threshold, the planned path is adjusted.

[0056] In a second aspect, the present application provides a path planning device, comprising:

[0057] An acquisition module is used to acquire a starting posture, a target posture, a preset exploration direction set and a grid map of a driving environment corresponding to the path planning, and use the starting posture as the current posture;

[0058] A processing module, used to determine the obstacle position closest to the current posture from the grid map, input the current posture, the exploration direction set and the obstacle position into a pre-trained exploration model, obtain an evaluation result of each exploration direction in the exploration direction set, and screen out multiple candidate exploration directions of the current posture from the exploration direction set according to the evaluation result;

[0059] The processing module is further used to determine multiple candidate next poses of the current pose according to the current pose and the multiple candidate exploration directions, and determine the remaining distance of each candidate next pose according to each candidate next pose and the target pose, wherein the remaining distance is the distance between the candidate next pose and the target pose;

[0060] The processing module is further used to select a next posture from a plurality of candidate next postures according to each of the remaining distances, and add the next posture to the planned path;

[0061] The processing module is further configured to update the current posture to the next posture when the next posture is outside the preset distance range of the target posture, and return to the step of determining the obstacle position closest to the current posture from the grid map;

[0062] The processing module is further used to connect the next posture and the target posture through an RS curve when the next posture is within a preset distance range of the target posture, and end the path planning when the connection is successful.

[0063] In a path planning device provided in this application:

[0064] a processing module, configured to extract a next position and a next posture from the next position, and extract a first distance between a grid occupied by the vehicle and the axle when the vehicle is located at the next position in the next posture from a pre-generated collision detection table;

[0065] The processing module is further used to determine, according to the first distance, a first grid range occupied by the vehicle when it is located at the next position in the next posture;

[0066] The processing module is further used to extract obstacle information located within the first grid range from the grid map;

[0067] If there is no obstacle information within the first grid range, the processing module is further used to add the next posture to the planned path;

[0068] The step of connecting the next posture and the target posture through an RS curve includes:

[0069] The processing module is further used to divide the RS curve into a plurality of continuous posture points;

[0070] The processing module is further used to extract the first position and the first posture from each of the posture points, and extract the second distance between the grid occupied by the vehicle and the axle when the vehicle is located at the first position in the first posture from a pre-generated collision detection table;

[0071] The processing module is further used to determine, according to the second distance, a second grid range occupied by the vehicle when the vehicle is located at the first position in the first posture;

[0072] The processing module is further used to extract obstacle information located within the second grid range from the grid map;

[0073] If there is no obstacle information within the second grid range of each of the RS curves, the processing module is further used to connect the next posture and the target posture through the RS curve.

[0074] In a path planning device provided in this application:

[0075] An acquisition module is used to acquire a training start posture and use the training start posture as a training current posture;

[0076] A processing module, used for determining the obstacle position closest to the training current posture from the grid map as the training obstacle position, so as to input the training current posture, the exploration direction set and the training obstacle position into the exploration model to obtain the target exploration direction in the exploration direction set;

[0077] The processing module is further used to determine the next posture of the training current posture according to the training current posture and the target exploration direction as the training next posture, and calculate the reward value of the training next posture according to the training next posture;

[0078] The processing module is also used to determine the total reward value according to the reward value of each posture on the planned path at the end of the path planning;

[0079] If the total reward value does not meet the preset convergence condition, the processing module is further used to adjust the exploration model and return to the step of determining the obstacle position closest to the current training posture from the grid map as the training obstacle position to perform the next round of iteration;

[0080] If the total reward value meets the preset convergence condition, the training ends.

[0081] In a path planning device provided in this application:

[0082] A processing module, used for determining the obstacle position closest to the training next posture from the grid map according to the training next posture;

[0083] The processing module is further used to determine the obstacle collision reward of the training next posture according to the obstacle position closest to the training next posture and the training next posture;

[0084] The processing module is further used to determine the remaining distance reward of the training next posture according to the training next posture and the target posture;

[0085] The processing module is further used to calculate the reward value of the training next posture according to the obstacle collision reward of the training next posture and the remaining distance reward of the training next posture;

[0086] The calculation formula for the reward value of the training next posture is as follows:

[0087]

[0088] Among them, r obs To train the obstacle collision reward for the next pose, r goal To train the remaining distance reward for the next posture, XC t+1 and YC t+1 is the 2D coordinate of the position included in the training next pose, XO t+1 and YO t+1 is the two-dimensional coordinate of the obstacle position closest to the training next pose, D is the preset obstacle detection radius, h reward is a coefficient greater than 0, h max 、h min are the maximum distance and minimum distance between each position in the grid map and the target position, respectively, t+1 To train the distance between the next pose and the target pose;

[0089] The total reward value is determined according to the reward value of each posture on the planned path, including:

[0090] An acquisition module is used to acquire a preset planning success reward, a preset planning failure reward, and a preset single exploration reward. When the path planning is successful, the planning success reward is greater than 0, and the planning failure reward is 0. When the path planning fails, the planning success reward is 0, the planning failure reward is less than 0, and the preset single exploration reward is less than 0.

[0091] The processing module is further used to determine a total reward value according to the planning success reward, the planning failure reward, the single exploration reward and the reward value of each posture;

[0092] The calculation formula of the total reward value is as follows:

[0093]

[0094] Among them, R is the total reward value, r terminated For the preset planning success reward, r truncated is the preset reward for planning failure, r t+1is the reward value of the t+1th posture, N is the number of postures included in the planned path during training, and t is an integer greater than or equal to 1 and less than or equal to N-1.

[0095] In a path planning device provided in this application:

[0096] A processing module, configured to extract one or more discrete pose pairs from the planned path;

[0097] For each discrete pose pair, a processing module is used to use two discrete poses in the discrete pose pair as two endpoints to fit an arc between the two endpoints;

[0098] A processing module is used to insert a posture on the arc according to a preset distance and add the posture to the planned path, wherein the inserted posture is located between two discrete postures of the discrete posture pair in sequence, and the posture in the inserted posture is consistent with the tangent direction of the arc.

[0099] In a path planning device provided in this application:

[0100] An acquisition module, used to acquire the angle change between adjacent postures on the planned path;

[0101] If at least one of the angle changes on the planned path is greater than or equal to a preset angle, the processing module is used to adjust the planned path, and the preset angle is determined according to the maximum turning direction of the vehicle.

[0102] In a path planning device provided in this application:

[0103] A processing module, used for determining a driving direction according to any two adjacent positions on the planned path;

[0104] The processing module is further used to determine the number of switches to the opposite direction on the planned path according to the multiple driving directions on the planned path;

[0105] If the number of opposite direction switches is greater than or equal to a preset number threshold, the processing module is further configured to adjust the planned path.

[0106] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0107] The memory stores computer-executable instructions;

[0108] The processor executes the computer-executable instructions stored in the memory to implement a path planning method as described in any one of the first aspects.

[0109] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement a path planning method as described in any one of the first aspects.

[0110] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements a path planning method as described in any one of the first aspects.

[0111] The path planning method, device, equipment, storage medium and program product provided by the present application first obtain the starting posture, target posture, preset exploration direction set and grid map of the driving environment corresponding to the path planning, and use the starting posture as the current posture, determine the obstacle position closest to the current posture from the grid map, so as to input the current posture, exploration direction set and obstacle position into the pre-trained exploration model, obtain the evaluation result of each exploration direction in the exploration direction set, and screen out multiple candidate exploration directions of the current posture from the exploration direction set according to the evaluation result. According to the current posture and multiple candidate exploration directions, multiple candidate next postures of the current posture are determined, and according to each candidate next posture and the target posture, the remaining distance of each candidate next posture is determined, and the remaining distance is the distance between the candidate next posture and the target posture. The next posture is selected from the multiple candidate next postures according to each remaining distance, and the next posture is added to the planned path. When the next posture is outside the preset distance range of the target posture, the current posture is updated to the next posture, and the step of determining the obstacle position closest to the current posture from the grid map is returned. When the next posture is within the preset distance range of the target posture, the next posture and the target posture are connected through the RS curve, and when the connection is successful, the path planning is ended. Compared with the prior art, it is necessary to traverse a large number of nodes to find the optimal path, which results in a long path planning time when the number of nodes is large, and ultimately reduces the transportation efficiency of the unmanned mine car. The present application selects several exploration directions with the best evaluation results based on the evaluation results of each exploration direction, and then performs subsequent path planning based on the exploration directions with the highest evaluation results, rather than performing path planning based on all exploration directions. The present application reduces the size of the search space, improves the search efficiency, and thereby reduces the path planning time and improves the transportation efficiency of the unmanned mine car. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0113] Figure 1 A schematic diagram of a flow chart of a first embodiment of a path planning method provided in an embodiment of the present application;

[0114] Figure 2 A schematic diagram of a flow chart of a second embodiment of a path planning method provided in an embodiment of the present application;

[0115] Figure 3 A schematic diagram of a flow chart of a third embodiment of a path planning method provided in an embodiment of the present application;

[0116] Figure 4 A schematic diagram of a flow chart of a fourth embodiment of a path planning method provided in an embodiment of the present application;

[0117] Figure 5 This is a schematic diagram of the path planning process for this application;

[0118] Figure 6 A path planning device provided in an embodiment of the present application;

[0119] Figure 7 A schematic diagram of the structure of an electronic device embodiment 1 provided in an embodiment of the present application.

[0120] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0121] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0123] In the prior art, the path planning algorithm searches for the optimal path by traversing a large number of nodes, which results in a very long planning time when there are too many directions to explore. This causes the actual planned path to be out of touch with the actual mining scene, ultimately resulting in low transportation efficiency of the unmanned mining vehicles.

[0124] Based on this, the inventive concept of this application is to provide a path planning method, which evaluates the evaluation results of each exploration direction and only selects several exploration directions with higher evaluation structures for subsequent exploration, thereby reducing the size of the search space, improving the search efficiency, reducing the path planning time, and improving the transportation efficiency of the unmanned mining car.

[0125] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0126] Figure 1 A schematic diagram of a flow chart of a first embodiment of a path planning method provided in an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0127] S101: Obtain a starting posture, a target posture, a preset exploration direction set and a grid map of a driving environment corresponding to the path planning, and use the starting posture as the current posture.

[0128] In the embodiment of the present application, a grid map of the mine car driving environment is first obtained, and the starting position (x start ,y start ,θ start ), the target position of the minecart (x end ,y end ,θ end ), the preset exploration direction set, and the starting pose is used as the current pose. start Indicates the horizontal coordinate of the minecart starting point, y start Indicates the vertical coordinate of the minecart at the start, θ start Indicates the starting orientation of the minecart; x end Indicates the horizontal coordinate when the minecart reaches the target position, y end Represents the ordinate when the minecart reaches the target position, θ end Represents the orientation angle of the minecart when it reaches the target position; the preset exploration direction set is obtained by the reinforcement learning process.

[0129] S102: Determine the obstacle position closest to the current posture from the grid map, input the current posture, exploration direction set and obstacle position into a pre-trained exploration model, obtain an evaluation result of each exploration direction in the exploration direction set, and screen out multiple candidate exploration directions for the current posture from the exploration direction set based on the evaluation results.

[0130] In the embodiment of the present application, after obtaining the starting posture, target posture, preset exploration direction set and grid map of the driving environment corresponding to the path planning, starting from the current posture, obstacle search is performed in the grid map, and then the obstacle position (X) closest to the current posture is determined from the grid map. obst ,Y obst ), where X obst The horizontal coordinate of the obstacle position, Y obst The ordinate represents the position of the obstacle. Subsequently, the current posture, exploration direction set and obstacle position are input into the pre-trained exploration model to obtain the evaluation result of each exploration direction in the exploration direction set, and then multiple candidate exploration directions of the current posture are screened out from the exploration direction set based on the evaluation result. The pre-trained exploration model will obtain the evaluation result of each exploration direction in the exploration direction set. In order to narrow the exploration scope and ensure the exploration effect, multiple exploration direction sets with better evaluation results are selected from the exploration direction set as multiple candidate exploration directions. Among them, the evaluation result is the Q value output by the pre-trained exploration model.

[0131] In step S102, the pre-trained exploration model is trained as follows:

[0132] Get the training start pose and use it as the training current pose.

[0133] In the embodiment of the present application, when training the exploration model, it is necessary to build an interactive environment, determine the state space, and set up a reward function, and then train the model by acquiring data. Therefore, first obtain the training starting posture, and use the training starting posture as the training current posture. The starting posture includes the initial horizontal coordinate X0, the initial vertical coordinate Y0, and the initial orientation angle Y of the mine car. aw0 .

[0134] The obstacle position closest to the current training posture is determined from the grid map as the training obstacle position, so that the current training posture, the exploration direction set and the training obstacle position are input into the exploration model to obtain the target exploration direction in the exploration direction set.

[0135] In the embodiment of the present application, the position of the obstacle closest to the training current posture is determined from the grid map and used as the training obstacle position. Then, the training current posture, the exploration direction set and the training obstacle position are input into the exploration model to obtain the target exploration direction from the exploration direction set.

[0136] Optionally, in order to prevent the overestimation caused by the bootstrap maximization problem when using the DQN network structure alone for training, the DQN network mistakenly believes that some suboptimal actions are optimal, thereby causing deviations in action selection, and ultimately leading to performance degradation of the network model. Figure 2 As shown is a schematic diagram of the exploration model structure of the present application. The exploration model of the embodiment of the present application is trained using a double Q learning network (Double Q-Network, DDQN), wherein the double Q learning network includes a DQN network and a target network, and the DQN network structure is the same as the target network structure, and the parameters of the target network structure are periodically copied from the DQN network structure parameters, and the parameters are kept unchanged during copying.

[0137] The input of the DQN network is the current state, and the output is the next exploration direction; the target network inputs the current state and the next exploration direction, and the output is the predicted value of the next exploration direction, that is, the Q value. The Q value is called the action value function, which indicates the expected total reward that the agent can obtain from the current state to the future state after taking an action and following a certain strategy in a given state. In short, the Q value is an indicator to evaluate the quality of a specific state and action pair.

[0138] Therefore, the current training posture, exploration direction set and training obstacle position are input into the exploration model. The exploration model calculates the Q value corresponding to each direction in the exploration direction set based on the input state information, and selects the direction corresponding to the maximum Q value as the target exploration direction.

[0139] According to the current training posture and the target exploration direction, the next posture of the current training posture is determined as the next training posture, and the reward value of the next training posture is calculated according to the next training posture.

[0140] In an embodiment of the present application, based on the training current posture and the acquired target exploration direction, the next posture of the training current posture is calculated as the training next posture, and the reward value of the next posture is calculated based on the training next posture.

[0141] Among them, the specific steps of calculating the reward value of the next training posture according to the next training posture are:

[0142] According to the training next pose, the obstacle position closest to the training next pose is determined from the grid map.

[0143] In an embodiment of the present application, when the next training posture is determined according to the current posture and the target exploration direction, the obstacle position closest to the next training posture is determined from the grid map according to the next training posture.

[0144] The obstacle collision reward for the training next pose is determined based on the nearest obstacle position of the training next pose and the training next pose.

[0145] In the embodiment of the present application, the obstacle collision reward of the training next posture is determined according to the training next posture and the obstacle position closest to the training next posture. The obstacle collision reward calculation formula is:

[0146]

[0147] Among them, r obs Indicates the obstacle collision reward value for training the next posture; XC t+1 Indicates the horizontal coordinate of the mine car posture for training the next posture; YC t+1 Indicates the vertical coordinate of the minecart posture for training the next posture; XO t+1 Indicates the horizontal coordinate of the obstacle position closest to the next training pose; YO t+1 Indicates the ordinate of the obstacle position closest to the next training pose; D represents the preset obstacle detection radius, which is a fixed value determined by the needs.

[0148] Based on the training next pose and the target pose, determine the remaining distance reward for training the next pose.

[0149] In the embodiment of the present application, the remaining distance reward for training the next posture is determined based on the training next posture and the target posture. The calculation formula of the remaining distance reward is:

[0150]

[0151] Among them, r goal Indicates the remaining distance reward value; h reward Represents the reward coefficient, a positive fixed value selected according to needs; h max It represents the maximum value of the heuristic distance of each point in the grid map relative to the target pose; h min Represents the minimum value of the heuristic distance of each point in the grid map relative to the target pose; d t+1 Indicates the heuristic distance of the next pose to be trained relative to the target pose. Optionally, the heuristic distance is Euclidean distance, Manhattan distance, or diagonal distance.

[0152] The reward value of the training next pose is calculated based on the obstacle collision reward of the training next pose and the remaining distance reward of the training next pose.

[0153] In the embodiment of the present application, the reward value for training the next posture is the sum of the obstacle collision reward for training the next posture and the remaining distance reward for training the next posture. The calculation formula for the reward value for training the next posture is as follows:

[0154]

[0155] At the end of path planning, the total reward value is determined based on the reward values ​​of each posture on the planned path.

[0156] In the embodiment of the present application, at the end of path planning, it is necessary to determine the total reward value based on the reward values ​​of each posture on the obtained path.

[0157] The total reward value is determined based on the reward values ​​of each posture on the planned path, including:

[0158] Get the preset planning success reward, preset planning failure reward, and preset single exploration reward. When the path planning is successful, the planning success reward is greater than 0, and the planning failure reward is 0. When the path planning fails, the planning success reward is 0, the planning failure reward is less than 0, and the preset single exploration reward is less than 0.

[0159] In the embodiment of the present application, in order to determine the total reward value, it is necessary to obtain the preset planning success reward, the preset planning failure reward, and the preset single exploration reward; wherein, when the path planning is successful, the planning success reward is greater than 0, and the planning failure reward is 0; when the path planning fails, the planning success reward is 0, and the planning failure reward is less than 0; the preset single exploration reward is less than 0. The planning success reward, planning failure reward, and single exploration reward can be adjusted according to actual needs. Optionally, the planning success reward r terminated 、Reward for planning failure truncated And a single exploration reward r step for:

[0160]

[0161] r step =-1

[0162] The total reward value is determined based on the planning success reward, planning failure reward, single exploration reward and the reward value of each posture.

[0163] In the embodiment of the present application, the calculation formula of the total reward value is as follows:

[0164]

[0165] Among them, R is the total reward value, r terminated For the preset planning success reward, r truncated is the preset reward for planning failure, r t+1 is the reward value of the t+1th posture, N is the number of postures included in the planned path during training, and t is an integer greater than or equal to 1 and less than or equal to N-1. Since the reward value of a posture is calculated based on the next posture, there is a reward value starting from the second posture.

[0166] If the total reward value does not meet the preset convergence condition, the exploration model is adjusted and the step of determining the obstacle position closest to the current training posture from the grid map as the training obstacle position is returned to perform the next round of iteration.

[0167] In the embodiment of the present application, if the total reward value R does not meet the preset convergence condition, the exploration model is adjusted, and the process of adjusting the exploration model includes but is not limited to: adjusting hyperparameters, such as learning rate, batch size, modifying reward function, etc. Then, the step of determining the obstacle position closest to the current training posture from the grid map as the training obstacle position is returned to iterate again.

[0168] If the total reward value meets the preset convergence condition, the training ends.

[0169] In the embodiment of the present application, if the final total reward value R meets the preset convergence condition, the training ends, and the exploration model at this time is considered to have good path planning capabilities.

[0170] S103: Determine multiple candidate next poses of the current pose according to the current pose and multiple candidate exploration directions, and determine the remaining distance of each candidate next pose according to each candidate next pose and the target pose, where the remaining distance is the distance between the candidate next pose and the target pose.

[0171] In the embodiment of the present application, multiple candidate next poses of the current pose are determined from the grid map according to the current pose and the multiple candidate exploration directions selected, and then the remaining distance between each candidate next pose and the target pose is calculated, where the remaining distance is the distance between the candidate next pose and the target pose.

[0172] S104: Selecting a next pose from a plurality of candidate next poses according to the remaining distances, and adding the next pose to the planned path.

[0173] In an embodiment of the present application, based on the calculated residual distance between each candidate next posture and the target posture, the candidate next posture corresponding to the smallest residual distance is selected as the next posture, and the next posture is added to the planned path.

[0174] Among them, a specific implementation method of adding the next posture to the planned path is:

[0175] A next position and a next posture are extracted from the next position, and a first distance between a grid occupied by the vehicle and the axle when the vehicle is located at the next position with the next posture is extracted from a pre-generated collision detection table.

[0176] In the embodiment of the present application, the pre-generation process of the collision detection table is as follows: a virtual mine car model is constructed in a digital environment according to the actual size and physical characteristics of the mine car. In order to determine the occupancy of the mine car in different postures, the mine car model needs to be rotated in the grid map, usually by changing the orientation angle Y of the vehicle model. aw To achieve this, simulate the steering process of the vehicle in the actual environment. As the minecart model rotates and moves in the grid map, all grid positions occupied by the minecart model are recorded. In order to simplify the calculation, the center of the rear axle of the minecart is selected as the reference point. For each occupied grid position, the distance between the grid position and the center of the rear axle of the minecart is recorded, and all the occupied grid positions and their distance information from the center of the rear axle of the minecart are organized into a collision detection table.

[0177] Therefore, the next position and the next posture are extracted from the next posture, and then the pre-generated collision detection table is queried to extract the first distance between the grid occupied by the vehicle in the next posture and the axle from the pre-generated collision detection table.

[0178] A first grid range occupied by the vehicle when it is located at a next position in a next posture is determined according to the first distance.

[0179] In the embodiment of the present application, the grid range occupied by the vehicle when it is in the next position in the next posture, that is, the first grid range, is determined according to the first distance between the grid occupied by the vehicle in the next posture and the axle.

[0180] Obstacle information within the first grid range is extracted from the grid map.

[0181] In the embodiment of the present application, after the first grid range is obtained, obstacle information within the first grid range is extracted from the grid map.

[0182] If there is no obstacle information within the first grid range, the next pose is added to the planned path.

[0183] In the embodiment of the present application, if there is no obstacle information within the first grid range, the mine car will not collide with the obstacle at the new position and posture, so the next posture is added to the planned path.

[0184] Connect the next pose and the target pose through the RS curve, including:

[0185] In the embodiment of the present application, the RS curve refers to the Reeds-Shepp curve, which was proposed by JA Reeds and LA Shepp in 1990 to solve the continuous and discontinuous curvature path planning problems. The RS curve is composed of several arcs with fixed radius and a straight line segment, and the radius of the arc is equal to the minimum turning radius of the vehicle. Among them, when the next posture is within the preset distance range of the target posture, the next posture and the target posture are connected by the RS curve. Optionally, the preset distance range of the target posture is 20 meters, that is, when the distance between the next posture and the target posture is within 20 meters, the next posture and the target posture are connected by the RS curve.

[0186] Divide the RS curve into multiple continuous pose points.

[0187] In the embodiment of the present application, the RS curve obtained by connection is divided into a plurality of continuous posture points, each of which represents a different position and direction on the curve.

[0188] A first position and a first posture are extracted from each posture point, and a second distance between a grid occupied by the vehicle and the axle when the vehicle is located at the first position in the first posture is extracted from a pre-generated collision detection table.

[0189] In an embodiment of the present application, the first posture and the first attitude are extracted from each posture point on the RS curve, and a pre-generated collision detection table is queried to query the second distance between the grid occupied by the vehicle and the axle when the vehicle is in the first position with the first posture.

[0190] A second grid range occupied by the vehicle when it is located at the first position in the first posture is determined according to the second distance.

[0191] In the embodiment of the present application, for each posture point on the RS curve, a second grid range occupied by the vehicle when it is located at the first position in the first posture is determined according to the second distance.

[0192] Obstacle information within the second grid range is extracted from the grid map.

[0193] In the embodiment of the present application, for each posture point on the RS curve, obstacle information within the second grid range is extracted from the grid map, that is, it is checked whether there is obstacle information within the second grid range.

[0194] If there is no obstacle information within the second grid range of each RS curve, the next pose and the target pose are connected through the RS curve.

[0195] In an embodiment of the present application, if there is no obstacle information within the second grid range on the RS curve, that is, the entire path is safe, the next posture and the target posture are connected through the RS curve to complete the path planning.

[0196] S105: When the next posture is outside the preset distance range of the target posture, the current posture is updated to the next posture, and the process returns to the step of determining the obstacle position closest to the current posture from the grid map.

[0197] In an embodiment of the present application, if the next posture is outside the preset distance range of the target posture, optionally, when the distance between the next posture and the target posture is greater than 20 meters, the current posture is updated to the next posture, and the position of the obstacle closest to the current posture is re-determined from the grid map based on the next posture after the update.

[0198] S106: When the next posture is within a preset distance range of the target posture, the next posture and the target posture are connected through the RS curve, and when the connection is successful, the path planning is ended.

[0199] In an embodiment of the present application, if the next posture is within a preset distance range of the target posture, optionally, when the preset distance range between the next posture and the target posture is less than or equal to 20 meters, the next posture and the target posture are connected by an RS curve, and when the connection is successful, the path planning is terminated to ensure the safety of the path and avoid the collision of the path with environmental obstacles due to the randomness of the strategy selection given by the exploration model.

[0200] In an embodiment of the present application, the starting posture, target posture, preset exploration direction set and grid map of the driving environment corresponding to the path planning are first obtained, and the starting posture is used as the current posture, and the obstacle position closest to the current posture is determined from the grid map, so that the current posture, exploration direction set and obstacle position are input into the pre-trained exploration model, and the evaluation result of each exploration direction in the exploration direction set is obtained, and multiple candidate exploration directions of the current posture are screened out from the exploration direction set according to the evaluation result. According to the current posture and multiple candidate exploration directions, multiple candidate next postures of the current posture are determined, and according to each candidate next posture and the target posture, the remaining distance of each candidate next posture is determined, and the remaining distance is the distance between the candidate next posture and the target posture. The next posture is selected from the multiple candidate next postures according to each remaining distance, and the next posture is added to the planned path. When the next posture is outside the preset distance range of the target posture, the current posture is updated to the next posture, and the step of determining the obstacle position closest to the current posture from the grid map is returned. When the next posture is within the preset distance range of the target posture, the next posture and the target posture are connected through the RS curve, and when the connection is successful, the path planning is ended. Compared with the prior art, it is necessary to traverse a large number of nodes to find the optimal path, which results in a long path planning time when the number of nodes is large, and ultimately reduces the transportation efficiency of the unmanned mine car. The present application selects several exploration directions with the best evaluation results based on the evaluation results of each exploration direction, and then performs subsequent path planning based on the exploration directions with the highest evaluation results, rather than performing path planning based on all exploration directions. The present application reduces the size of the search space, improves the search efficiency, and thereby reduces the path planning time and improves the transportation efficiency of the unmanned mine car.

[0201] Figure 2 The figure is a flow chart of a second embodiment of a path planning method provided by the present application. Figure 1 On the basis of the embodiment shown, after the path planning is completed in step S106, the method further includes:

[0202] S201: Extract one or more discrete pose pairs from the planned path.

[0203] In an embodiment of the present application, after the path planning is completed, a plurality of discrete pose points are output. Optionally, two adjacent discrete pose points are taken as a discrete pose pair, thereby extracting a plurality of discrete pose pairs from the planned path.

[0204] S202: For each discrete pose pair, two discrete poses in the discrete pose pair are used as two endpoints to fit an arc between the two endpoints.

[0205] In an embodiment of the present application, for each discrete pose pair, the two discrete poses in the discrete pose pair are used as two endpoints, and then an arc is fitted for each discrete pose pair according to the arc interpolation method to obtain an arc between the two endpoints.

[0206] S203: inserting a pose on the arc according to a preset distance and adding the pose to the planned path, wherein the inserted pose is located between two discrete poses of the discrete pose pair in sequence, and the pose in the inserted pose must be consistent with the tangent direction of the arc.

[0207] In an embodiment of the present application, a preset distance is first determined, and then a posture is inserted on the arc between the discrete posture pairs based on each discrete posture pair, its corresponding arc, and the preset distance, and the posture in the inserted posture needs to make the tangent direction of the arc consistent.

[0208] In the embodiment of the present application, after the path planning is completed, one or more discrete pose pairs are extracted from the planned path. For each discrete pose pair, two discrete poses in the discrete pose pair are used as two endpoints to fit the arc between the two endpoints, and the pose is inserted on the arc according to the preset distance, and the pose is added to the planned path. The inserted pose is located between the two discrete poses of the discrete pose pair in order, and the pose in the inserted pose must be consistent with the tangent direction of the arc. The planned path is refined and the smoothness and feasibility of the planned path are guaranteed.

[0209] Figure 3 FIG. 1 is a flow chart of a third embodiment of a path planning method provided in an embodiment of the present application. Figure 3 As shown, the method comprises the following steps:

[0210] S301: Obtain the angle change between adjacent postures on the planned path.

[0211] In an embodiment of the present application, the angle change between adjacent postures on the planned path is obtained, wherein the angle change between adjacent postures is the difference between the orientation angles of the two postures.

[0212] S302: If at least one angle change on the planned path is greater than or equal to a preset angle, the planned path is adjusted, where the preset angle is determined based on the maximum turning direction of the vehicle.

[0213] In the embodiment of the present application, if the obtained change in angle between adjacent positions is greater than or equal to a preset angle, the planned path is adjusted, and optionally, additional positions are inserted between adjacent positions to smooth the transition angle change. The preset angle is determined according to the maximum turning direction of the vehicle.

[0214] In the embodiment of the present application, by obtaining the angle change between adjacent postures on the planned path, if at least one angle change on the planned path is greater than or equal to the preset angle, the planned path is adjusted. To check the smoothness of the planned path, if it is found that at least one angle change on the planned path is greater than or equal to the maximum turning direction of the vehicle, it indicates that the part of the path is too steep and needs to be readjusted to ensure that the vehicle can drive smoothly. Checking and adjusting the planned path helps to improve the feasibility and safety of the path.

[0215] Figure 4 FIG. 1 is a flow chart of a fourth embodiment of a path planning method provided in an embodiment of the present application. Figure 4 As shown, the method comprises the following steps:

[0216] S401: Determine the driving direction according to any two adjacent positions on the planned path.

[0217] In the embodiment of the present application, the driving direction is first determined according to any two adjacent positions on the planned path. Optionally, if the latter position has a positive displacement in the x-axis direction relative to the previous position, then the driving direction is eastward.

[0218] S402: Determine the number of switches to the opposite direction on the planned path according to the multiple driving directions on the planned path.

[0219] In the embodiment of the present application, after the driving direction between each adjacent posture is determined, the entire planned path is traversed to check the switching of the driving direction. For example, if the driving direction suddenly changes from east to west, it is recorded as a switch in the opposite direction. The number of switches in the opposite direction on the entire planned path is accumulated and calculated.

[0220] S403: If the number of opposite direction switches is greater than or equal to a preset number threshold, the planned path is adjusted.

[0221] In an embodiment of the present application, if the number of opposite direction switches is greater than or equal to a preset number threshold, it indicates that there are too many sharp turns or unnecessary direction changes in the planned path, and therefore the planned path needs to be adjusted.

[0222] In the embodiment of the present application, the driving direction is determined according to any two adjacent positions on the planned path, and the number of switches in the opposite direction on the planned path is determined according to the multiple driving directions on the planned path. If the number of switches in the opposite direction is greater than or equal to the preset number threshold, it means that there are too many sharp turns or unnecessary direction changes in the path planning, which will affect the driving efficiency and cause driving safety. Therefore, the planned path needs to be adjusted. To reduce unnecessary direction changes to improve driving efficiency and driving safety.

[0223] Figure 5 This is a schematic diagram of the path planning process for this application, such as Figure 5 As shown in the figure, firstly, the starting point and end point of the path planning and the grid map are obtained, and then forward exploration is performed according to the starting point and end point of the path planning and the grid map. The forward exploration outputs the corresponding child node, and then when the child node is within the preset distance range of the end point, the current child node and the end point path are connected through the RS curve. If the RS curve connection is successful, the original path is output. If the RS curve connection fails, the forward exploration is performed again. After the original path is output through the RS connection, the original path needs to be processed, and finally the explored path is output. Forward exploration includes inputting state variables, so that the exploration model performs forward propagation, and then outputs the Q value of all exploration direction sets, selects the three exploration branches with the largest Q value, and then reduces the size of the search space and improves the search efficiency. Finally, for the three selected exploration branches, the corresponding child nodes are calculated, and the child nodes are connected to the end point through RS to ensure that the explored path is safe.

[0224] Figure 6 A path planning device provided in an embodiment of the present application. A path planning device provided in the present application includes: an acquisition module 61 and a processing module 62; wherein,

[0225] The acquisition module 61 is used to obtain the starting posture, target posture, preset exploration direction set and grid map of the driving environment corresponding to the path planning, and use the starting posture as the current posture; the processing module 62 is used to determine the obstacle position closest to the current posture from the grid map, so as to input the current posture, exploration direction set and obstacle position into the pre-trained exploration model, obtain the evaluation result of each exploration direction in the exploration direction set, and screen out multiple candidate exploration directions of the current posture from the exploration direction set according to the evaluation result; the processing module 62 is also used to determine multiple candidate next postures of the current posture according to the current posture and multiple candidate exploration directions, and select the candidate next postures according to each candidate next posture. The processing module 62 is further used to select a next posture from multiple candidate next postures according to the residual distances, and add the next posture to the planned path; the processing module 62 is further used to update the current posture to the next posture when the next posture is outside the preset distance range of the target posture, and return to the step of determining the position of the obstacle closest to the current posture from the grid map; the processing module 62 is further used to connect the next posture and the target posture through the RS curve when the next posture is within the preset distance range of the target posture, and end the path planning when the connection is successful.

[0226] Figure 7A schematic diagram of the structure of an electronic device embodiment 1 provided in the present application. Figure 7 The electronic device provided in this embodiment includes: at least one processor 71 and a memory 72. The processor 71 and the memory 72 are connected via a bus 73.

[0227] The specific implementation process of the processor 71 can be found in the above-mentioned method embodiment. Its implementation principle and technical effects are similar, and will not be repeated here in this embodiment.

[0228] In the above Figure 7 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0229] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0230] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0231] The present application also provides a computer-readable storage medium on which computer execution instructions are stored. When a processor executes the computer execution instructions, a path planning method as described above is implemented.

[0232] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The computer-readable storage medium may be any available medium that can be accessed by a general or special purpose computer.

[0233] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0234] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0235] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0236] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0237] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0238] The present application also provides a computer program product, including a computer program, which is used to implement the above path planning method when executed by a processor.

[0239] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0240] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0241] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0242] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0243] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0244] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0245] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0246] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0247] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A path planning method, Its characteristics are: include: Obtaining the starting posture, target posture, preset exploration direction set and grid map of the driving environment corresponding to the path planning, and taking the starting posture as the current posture; Determine the obstacle position closest to the current posture from the grid map, input the current posture, the exploration direction set and the obstacle position into a pre-trained exploration model, obtain an evaluation result of each exploration direction in the exploration direction set, and screen out multiple candidate exploration directions of the current posture from the exploration direction set according to the evaluation result; Determine multiple candidate next poses of the current pose according to the current pose and the multiple candidate exploration directions, and determine the remaining distance of each candidate next pose according to each candidate next pose and the target pose, wherein the remaining distance is the distance between the candidate next pose and the target pose; Selecting a next pose from a plurality of candidate next poses according to each of the remaining distances, and adding the next pose to the planned path; When the next posture is outside the preset distance range of the target posture, updating the current posture to the next posture, and returning to the step of determining the obstacle position closest to the current posture from the grid map; When the next posture is within a preset distance range of the target posture, the next posture and the target posture are connected through an RS curve, and when the connection is successful, the path planning is ended.

2. The method according to claim 1, It is characterized in that The adding the next posture to the planned path comprises: Extracting a next position and a next posture from the next position, and extracting a first distance between a grid occupied by the vehicle and the axle when the vehicle is located at the next position in the next posture from a pre-generated collision detection table; Determine, according to the first distance, a first grid range occupied by the vehicle when it is located at the next position in the next posture; Extracting obstacle information within the first grid range from the grid map; If there is no obstacle information within the first grid range, adding the next posture to the planned path; The step of connecting the next posture and the target posture through an RS curve includes: Dividing the RS curve into a plurality of continuous posture points; Extracting a first position and a first posture from each of the posture points, and extracting a second distance between a grid occupied by the vehicle and the axle when the vehicle is located at the first position in the first posture from a pre-generated collision detection table; Determine, according to the second distance, a second grid range occupied by the vehicle when it is located at the first position in the first posture; Extracting obstacle information within the second grid range from the grid map; If there is no obstacle information within the second grid range of each of the RS curves, the next posture and the target posture are connected through the RS curve.

3. The method according to claim 1, Its characteristics are: The pre-training process of the exploration model includes: Obtaining a training start posture, and using the training start posture as a training current posture; Determine the obstacle position closest to the training current posture from the grid map as the training obstacle position, so as to input the training current posture, the exploration direction set and the training obstacle position into the exploration model to obtain the target exploration direction in the exploration direction set; According to the training current posture and the target exploration direction, determine the next posture of the training current posture as the training next posture, and calculate the reward value of the training next posture according to the training next posture; At the end of path planning, the total reward value is determined based on the reward values ​​of each posture on the planned path; If the total reward value does not meet the preset convergence condition, the exploration model is adjusted, and the step of returning to the step of determining the obstacle position closest to the current training posture from the grid map as the training obstacle position is performed to perform the next round of iteration; If the total reward value meets the preset convergence condition, the training ends.

4. The method according to claim 3, It is characterized in that The step of calculating the reward value of the next posture according to the training next posture comprises: Determining the obstacle position closest to the training next posture from the grid map according to the training next posture; Determining an obstacle collision reward for the training next pose according to the obstacle position closest to the training next pose and the training next pose; Determining a remaining distance reward for the training next posture according to the training next posture and the target posture; Calculating a reward value of the training next posture according to an obstacle collision reward of the training next posture and a remaining distance reward of the training next posture; The calculation formula for the reward value of the training next posture is as follows: Among them, r obs To train the obstacle collision reward for the next pose, r goal To train the remaining distance reward for the next posture, XC t+1 and YC t+1 is the 2D coordinate of the position included in the training next pose, XO t+1 and YO t+1 is the two-dimensional coordinate of the obstacle position closest to the next pose for training, D is the preset obstacle detection radius, h is reward is a coefficient greater than 0, h max 、h min are the maximum distance and minimum distance between each position in the grid map and the target position, respectively, t+1 To train the distance between the next pose and the target pose; The total reward value is determined according to the reward value of each posture on the planned path, including: Obtain the preset planning success reward, the preset planning failure reward, and the preset single exploration reward. When the path planning is successful, the planning success reward is greater than 0, and the planning failure reward is 0. When the path planning fails, the planning success reward is 0, the planning failure reward is less than 0, and the preset single exploration reward is less than 0. Determine a total reward value according to the planning success reward, the planning failure reward, the single exploration reward and the reward value of each of the postures; The calculation formula of the total reward value is as follows: Among them, R is the total reward value, r terminated For the preset planning success reward, r truncated is the preset reward for planning failure, r t+1 is the reward value of the t+1th posture, N is the number of postures included in the planned path during training, and t is an integer greater than or equal to 1 and less than or equal to N-1.

5. The method according to claim 1, It is characterized in that After the path planning is completed, the method further includes: Extracting one or more discrete pose pairs from the planned path; For each discrete pose pair, two discrete poses in the discrete pose pair are used as two endpoints to fit an arc between the two endpoints; A posture is inserted on the arc according to a preset distance, and the posture is added to the planned path, the inserted posture is located between two discrete postures of the discrete posture pair in sequence, and the posture in the inserted posture is consistent with the tangent direction of the arc.

6. The method according to claim 1, It is characterized in that The method further comprises: Obtaining the angle change between adjacent postures on the planned path; If at least one of the angle changes on the planned path is greater than or equal to a preset angle, the planned path is adjusted, and the preset angle is determined according to the maximum turning direction of the vehicle.

7. The method according to any one of claims 1 to 6, It is characterized in that The method further comprises: Determine the driving direction according to any two adjacent positions on the planned path; Determining the number of switches to the opposite direction on the planned path according to the plurality of driving directions on the planned path; If the number of opposite direction switches is greater than or equal to a preset number threshold, the planned path is adjusted.

8. A path planning device, It is characterized in that include: An acquisition module is used to acquire a starting posture, a target posture, a preset exploration direction set and a grid map of a driving environment corresponding to the path planning, and use the starting posture as the current posture; A processing module, used to determine the obstacle position closest to the current posture from the grid map, input the current posture, the exploration direction set and the obstacle position into a pre-trained exploration model, obtain an evaluation result of each exploration direction in the exploration direction set, and screen out multiple candidate exploration directions of the current posture from the exploration direction set according to the evaluation result; The processing module is further used to determine multiple candidate next poses of the current pose according to the current pose and the multiple candidate exploration directions, and determine the remaining distance of each candidate next pose according to each candidate next pose and the target pose, wherein the remaining distance is the distance between the candidate next pose and the target pose; The processing module is further used to select a next posture from a plurality of candidate next postures according to each of the remaining distances, and add the next posture to the planned path; The processing module is further configured to update the current posture to the next posture when the next posture is outside the preset distance range of the target posture, and return to the step of determining the obstacle position closest to the current posture from the grid map; The processing module is further used to connect the next posture and the target posture through an RS curve when the next posture is within a preset distance range of the target posture, and end the path planning when the connection is successful.

9. An electronic device, It is characterized in that include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

11. A computer program product, It is characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images