Path planning method and device of unmanned vehicle, electronic equipment and medium

Abstract

The application provides a path planning method and device for an unmanned vehicle, electronic equipment and a medium. The path planning method comprises: tracking an obstacle according to obstacle information, predicting a motion state of the obstacle, and obtaining prediction information; fusing obstacle information obtained by a plurality of devices according to the prediction information, and determining a target obstacle from the fused obstacle information; predicting an obstacle motion path of the target obstacle in a future prediction time period according to the motion state of the target obstacle; and planning a driving path for the unmanned vehicle to avoid the obstacle based on the obstacle motion path. The technical solution provided by the application can obtain more accurate obstacles after fusion through prediction information, thereby planning an obstacle avoidance path for the unmanned vehicle, ensuring real-time performance, improving the stability of the target tracking process, and improving the accuracy of the obstacle avoidance path planning for the unmanned vehicle.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a path planning method, device, electronic device and medium for autonomous vehicles. Background Technology

[0002] With the continuous development of intelligent vehicles, their need for perception of the surrounding environment is increasing. Among environmental perception algorithms, multi-target tracking algorithms are an important component, providing necessary environmental information for intelligent vehicles' trajectory planning and decision-making. Moreover, LiDAR has the characteristics of high perception accuracy and strong adaptability to lighting and weather, and has received widespread attention from academia and industry.

[0003] Currently, tracking algorithms mainly match the target detection results of consecutive frames to achieve continuous perception of the target's state. However, due to environmental factors such as rainy weather, the perception performance of lidar decreases, reducing the stability of target detection and affecting the target tracking process, resulting in inaccurate obstacle avoidance paths planned by autonomous vehicles. Therefore, how to improve the stability of the target tracking process while ensuring real-time performance, and thus improve the accuracy of obstacle avoidance paths for autonomous vehicles, has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a path planning method, device, electronic device and storage medium for autonomous vehicles, which can obtain more accurate obstacles by fusing predictive information, predict the movement path of the obstacle according to the movement state of the obstacle, and thus plan the obstacle avoidance path of the autonomous vehicle, realize the process of stable tracking of multiple targets in complex environments, improve the stability of the target tracking process while ensuring real-time performance, and accurately plan the obstacle avoidance path based on the stable target tracking process, thereby improving the accuracy of obstacle avoidance path planning for autonomous vehicles.

[0005] This application mainly includes the following aspects:

[0006] In a first aspect, embodiments of this application provide a path planning method for an autonomous vehicle, wherein the autonomous vehicle is equipped with different obstacle scanning devices, and the path planning method includes:

[0007] The obstacle is tracked based on obstacle information, and the motion state of the obstacle is predicted to obtain prediction information; wherein, the obstacle information includes information obtained by at least one obstacle scanning device; the motion state includes speed information and distance information;

[0008] Based on the predicted information, the obstacle information obtained by multiple devices is fused, and the target obstacle is determined from the fused obstacle information;

[0009] Based on the motion state of the target obstacle, the obstacle's motion path within a future predicted time period is predicted;

[0010] Based on the obstacle's movement path, a driving path is planned for the autonomous vehicle to avoid the obstacle.

[0011] Furthermore, the target obstacle is identified through the following steps:

[0012] If the prediction information is generated by lidar, then wait to obtain the lidar obstacle information and millimeter-wave radar obstacle information of the obstacle at the next moment;

[0013] If the prediction information is generated by millimeter-wave radar, then wait to obtain the lidar obstacle information of the obstacle at the next moment;

[0014] Based on the obtained obstacle information, the target obstacle is identified.

[0015] Furthermore, the motion state of the target obstacle is determined through the following steps:

[0016] Based on the similarity between the characteristics of the target obstacle and each tracked target in the tracking list, the target obstacle is matched with each tracked target in the tracking list, and the motion state of the tracked target corresponding to the target obstacle under the matching relationship is obtained from the tracking list at the previous moment; wherein, the tracking list stores the motion state of each tracked target at the previous moment.

[0017] Based on the motion state of the tracked target described in the previous moment, the motion state of the tracked target at the current moment is predicted to obtain the predicted value of the motion state at the current moment.

[0018] The predicted value of the current motion state is corrected to determine the current motion state of the target obstacle. The motion state of the next moment is predicted based on the current motion state, and the previous motion state of the tracked target in the tracking list is updated to the current motion state.

[0019] Furthermore, the step of correcting the predicted value of the current motion state to determine the current motion state of the target obstacle includes:

[0020] The target's motion state at the current moment is observed by multiple sensors. The predicted value is then corrected based on the observed value, and the corrected predicted value is determined as the target's motion state at the current moment.

[0021] Furthermore, the motion state of the tracked target corresponding to the target obstacle under the matching relationship at the previous moment is obtained through the following steps:

[0022] Determine the correlation matrix between the list of target obstacles and the tracking list;

[0023] Based on the correlation matrix, obtain the corresponding bipartite graph;

[0024] The bipartite graph is divided into multiple subgraphs according to a preset splitting threshold.

[0025] For each subgraph, a matching process is performed between the target obstacle and the tracked target; wherein, the target obstacle is matched with at most one tracked target, and the tracked target is matched with at most one target obstacle.

[0026] Based on the feature similarity between the target obstacle and the tracked target, it is determined whether the match is successful;

[0027] If the feature similarity between the target obstacle and the tracked target is greater than a preset threshold, the match is determined to be successful. Then, the motion state of the tracked target corresponding to the target obstacle at the previous moment is obtained from the matching relationship with the largest number of matches.

[0028] Furthermore, the matching relationship with the largest number of matches is determined through the following steps:

[0029] If the target object matched by the target obstacle has been matched by other target obstacles, then the matching path is reversed to determine whether other target obstacles matched with the target object have a next matching target object;

[0030] If present, match other target obstacles with the next matching tracking target, match the target obstacle with the tracking target, increment the matching count, and update the matching relationship;

[0031] If not, then check if the target obstacle has a next matching tracking target. If it does, then match the target obstacle with the next matching tracking target and increment the matching number by one.

[0032] Furthermore, after determining whether a match is successful, the path planning method further includes:

[0033] If the feature similarity between the target obstacle and the tracking target is less than a preset threshold, it is determined that the match is unsuccessful. Then, it is detected whether the unmatched target obstacle is appearing for the first time. If so, the target obstacle is added as a new tracking target to the tracking list, and the motion state of the target obstacle is predicted at the next moment.

[0034] If not, detect whether the tracked target has disappeared at the current moment. If the tracked target has disappeared at the current moment, predict the motion state of the tracked target, do not update the motion state, and increment the number of times only prediction is performed without updating.

[0035] If the number of times the prediction is made but not updated has reached a preset number, then the target will no longer be tracked and will be removed from the tracking list.

[0036] Furthermore, after determining the corrected predicted value as the current motion state of the tracking target, the path planning method further includes:

[0037] Based on the point cloud data information of the tracked target, determine whether the tracked target is a tracked target after the fusion of lidar obstacle information and millimeter-wave radar obstacle information;

[0038] If it is not a fused result, then obstacle information of the tracked target is obtained from the lidar scanning device and the millimeter-wave radar scanning device and fused.

[0039] Secondly, embodiments of this application also provide a path planning device for an autonomous vehicle, the path planning device comprising:

[0040] The acquisition module is used to track the obstacle based on obstacle information, predict the motion state of the obstacle, and acquire prediction information; wherein, the obstacle information includes information acquired by at least one obstacle scanning device; the motion state includes speed information and distance information;

[0041] The fusion module is used to fuse the obstacle information obtained by multiple devices based on the prediction information, and determine the target obstacle from the fused obstacle information;

[0042] The prediction module is used to predict the movement path of the target obstacle within a future predicted time period based on the movement state of the target obstacle.

[0043] The path planning module is used to plan the driving path of the autonomous vehicle to avoid obstacles based on the movement path of the obstacles.

[0044] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the path planning method for an autonomous vehicle as described above are performed.

[0045] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the path planning method for an autonomous vehicle as described above.

[0046] This application provides a path planning method, apparatus, electronic device, and storage medium for an autonomous vehicle. The path planning method includes: tracking obstacles based on obstacle information, predicting the motion state of the obstacles, and obtaining prediction information; wherein the obstacle information includes information acquired by at least one obstacle scanning device; the motion state includes speed information and distance information; fusing the obstacle information acquired by multiple devices based on the prediction information, and determining a target obstacle from the fused obstacle information; predicting the obstacle's motion path within a future predicted time period based on the target obstacle's motion state; and planning a driving path for the autonomous vehicle to avoid the obstacles based on the obstacle's motion path.

[0047] Thus, the technical solution provided in this application can obtain more accurate obstacles after fusion through predictive information, predict the movement path of the obstacle based on its movement state, and plan the obstacle avoidance path of the autonomous vehicle. This enables stable tracking of multiple targets in complex environments, improves the stability of the target tracking process while ensuring real-time performance, and allows for accurate obstacle avoidance path planning based on a stable target tracking process, thereby improving the accuracy of obstacle avoidance path planning for autonomous vehicles.

[0048] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart of a path planning method for an autonomous vehicle provided in an embodiment of this application is shown;

[0051] Figure 2 A flowchart illustrating another path planning method for autonomous vehicles provided in an embodiment of this application is shown;

[0052] Figure 3A schematic diagram of the target tracking algorithm provided in an embodiment of this application is shown;

[0053] Figure 4 This illustration shows one of the structural schematic diagrams of a path planning device for an autonomous vehicle provided in an embodiment of this application;

[0054] Figure 5 This is a second schematic diagram of the structure of a path planning device for an autonomous vehicle provided in an embodiment of this application;

[0055] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0057] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0058] In order to enable those skilled in the art to use the content of this application, and in combination with the specific application scenario of "path planning for autonomous vehicles", the following implementation methods are provided. For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application.

[0059] The methods, apparatus, electronic devices, or computer-readable storage media described in this application can be applied to any scenario requiring path planning. This application does not limit specific application scenarios. Any scheme using the path planning method, apparatus, electronic device, and storage media for autonomous vehicles provided in this application is within the protection scope of this application.

[0060] It is worth noting that with the continuous development of intelligent vehicles, their need for perception of the surrounding environment is increasing. Among environmental perception algorithms, multi-target tracking algorithms are an important component, providing necessary environmental information for intelligent vehicles' trajectory planning and decision-making. Moreover, LiDAR has the characteristics of high perception accuracy and strong adaptability to lighting and weather, and has received widespread attention from academia and industry.

[0061] Currently, tracking algorithms mainly match the target detection results of consecutive frames to achieve continuous perception of the target's state. However, due to environmental factors such as rainy weather, the perception performance of lidar deteriorates, affecting the stability of target detection. Therefore, how to improve the stability of the target tracking process while ensuring real-time performance has become an urgent problem to be solved.

[0062] Based on this, this application proposes a path planning method, device, electronic device, and storage medium for an autonomous vehicle. The path planning method includes: tracking the obstacle based on obstacle information, predicting the motion state of the obstacle, and obtaining prediction information; wherein the obstacle information includes information obtained by at least one obstacle scanning device; the motion state includes speed information and distance information; based on the prediction information, fusing the obstacle information obtained by multiple devices, and determining the target obstacle from the fused obstacle information; based on the motion state of the target obstacle, predicting the obstacle motion path of the target obstacle within a future predicted time period; and based on the obstacle motion path, planning a driving path for the autonomous vehicle to avoid the obstacle.

[0063] Thus, the technical solution provided in this application can obtain more accurate obstacles after fusion through predictive information, predict the movement path of the obstacle based on its movement state, and plan the obstacle avoidance path of the autonomous vehicle. This enables stable tracking of multiple targets in complex environments, improves the stability of the target tracking process while ensuring real-time performance, and allows for accurate obstacle avoidance path planning based on a stable target tracking process, thereby improving the accuracy of obstacle avoidance path planning for autonomous vehicles.

[0064] To facilitate understanding of this application, the technical solutions provided in this application will be described in detail below with reference to specific embodiments.

[0065] Please see Figure 1 , Figure 1 This is a flowchart illustrating a path planning method for an autonomous vehicle provided in an embodiment of this application. The autonomous vehicle is equipped with various obstacle scanning devices, such as... Figure 1 As shown, the path planning method includes:

[0066] S101. Track the obstacle based on the obstacle information, predict the movement state of the obstacle, and obtain prediction information;

[0067] In this step, the obstacle information includes information acquired by at least one obstacle scanning device; the motion state includes speed information and distance information; for example, the obstacle information scanned by the lidar device is processed by point cloud filtering, removal of ground and suspended objects, segmentation and clustering to determine multiple obstacles, and the obstacle is tracked and the motion state of the obstacle is predicted.

[0068] S102. Based on the predicted information, the obstacle information obtained by multiple devices is fused, and the target obstacle is determined from the fused obstacle information.

[0069] It should be noted that for the specific steps to identify the target obstacle, please refer to [link / reference needed]. Figure 2 , Figure 2 A flowchart illustrating another path planning method for autonomous vehicles provided in this application embodiment is shown below. Figure 2 As shown, the target obstacle is identified through the following steps:

[0070] S201. If the prediction information is generated by lidar, then wait to obtain the lidar obstacle information and millimeter-wave radar obstacle information of the obstacle at the next moment.

[0071] S202. If the prediction information is generated by millimeter-wave radar, then wait to obtain the lidar obstacle information of the obstacle at the next moment.

[0072] S203. Based on the obtained obstacle information, identify the target obstacle.

[0073] In this step, the source of the obstacle information is determined based on the predicted motion state of the obstacle at the current moment. Here, the point cloud data in the obstacle information has the identifiers of each scanning device. If the identifier indicates that the obstacle information belongs to LiDAR, then at the next moment, the LiDAR obstacle information and millimeter-wave radar obstacle information of the obstacle are acquired. If the identifier indicates that the obstacle information belongs to millimeter-wave radar, then at the next moment, the LiDAR obstacle information of the obstacle is acquired. Based on the acquired obstacle information, the more closely and accurately fused target obstacle is determined.

[0074] S103. Based on the motion state of the target obstacle, predict the obstacle's movement path within a future predicted time period;

[0075] It should be noted that the motion state of the target obstacle is determined through the following steps:

[0076] S1031. Based on the similarity between the characteristics of the target obstacle and each tracking target in the tracking list, the target obstacle is matched with each tracking target in the tracking list, and the motion state of the tracking target corresponding to the target obstacle under the matching relationship is obtained from the tracking list at the previous moment.

[0077] In this step, the tracking list stores the motion state of each tracked target at the previous moment;

[0078] It should be noted that the motion state of the tracked target corresponding to the target obstacle under the matching relationship at the previous moment is obtained through the following steps:

[0079] 1) Determine the correlation matrix between the list of target obstacles and the tracking list;

[0080] In this step, based on different preset feature metrics, different normalized distances are calculated between the target obstacle information detected by different devices and the tracking target information in the existing tracking list, thereby constructing the correlation matrix between each element of the two lists; the lidar uses the point cloud belonging to the target obstacle, and the millimeter-wave radar uses the detected direction of the target obstacle, and the obtained feature metrics have all been normalized to the probability interval of [0, 1].

[0081] The preset feature metrics include:

[0082] Distance metric: Used to measure the similarity in distance between the projections of various target obstacles in a specific coordinate system;

[0083] Shape metric: Used to measure the similarity of the projections of each target obstacle in a specific coordinate system onto the length, width, and size of the bounding box;

[0084] Velocity metric: used to measure the similarity in motion velocity of the projections of various target obstacles in a specific coordinate system;

[0085] Apparent metrics: used to measure the similarity of the projections of each target obstacle in a specific coordinate system onto the sensor-specific properties (such as the histogram of an optoelectronic camera, the number of point clouds in a lidar, or the signal-to-noise ratio of a millimeter-wave radar).

[0086] 2) Obtain the corresponding bipartite graph based on the correlation matrix;

[0087] For example, bipartite graph matching based on the Hungarian algorithm can be used. After obtaining the correlation matrix, a bipartite graph is constructed for the target obstacles of each sensor at the current time and the tracked targets tracked in the previous time step. One side of the bipartite graph consists of the nodes of the detected target obstacles, and the other side consists of the tracked target nodes in the tracking list; the edges between the nodes on both sides have non-negative costs, the values ​​of which are the feature metrics calculated in the previous step. Therefore, the matrix representation of this bipartite graph is the correlation matrix calculated in the previous step.

[0088] 3) Divide the bipartite graph into multiple subgraphs according to a preset splitting threshold;

[0089] In this step, the optimal match from the target obstacle to the tracked target can be found by minimizing the distance loss. This assignment problem can be solved in O(n log n) time. 3 It can be solved in polynomial time, and the original bipartite graph is further accelerated by using a pre-defined threshold to split it into several smaller subgraphs.

[0090] 4) For each sub-graph, perform matching between target obstacles and tracking targets; wherein, the target obstacle is matched with at most one tracking target, and the tracking target is matched with at most one target obstacle;

[0091] For example, before matching, the bipartite graph is initialized so that all edges point from the target obstacle S to the tracking target T, and the potential y is 0 everywhere.

[0092] 5) Determine whether a match is successful based on the feature similarity between the target obstacle and the tracked target;

[0093] It should be noted that, after determining whether a match is successful, the path planning method further includes:

[0094] (1) If the feature similarity between the target obstacle and the tracking target is less than a preset threshold, it is determined that the match is unsuccessful. Then, it is detected whether the unmatched target obstacle is appearing for the first time. If so, the target obstacle is added to the tracking list as a new tracking target, and the motion state of the target obstacle is predicted at the next moment.

[0095] In this step, if there are uncovered nodes in the bipartite graph, that is, unmatched target obstacles or tracking targets, for unmatched target obstacles, it is necessary to detect whether the target obstacle is appearing for the first time. If it is appearing for the first time, it is added to the tracking list as a new tracking target, and the motion state is predicted directly at the next moment.

[0096] (2) If not, detect whether the tracked target has disappeared at the current moment. If the tracked target has disappeared at the current moment, predict the motion state of the tracked target, do not update the motion state, and increment the number of times only prediction is performed without updating.

[0097] In this step, for unmatched tracking targets, it means that the tracking target existed in the previous time step but disappeared in the current time step. In this case, only prediction is made for the tracking target in the current time step without updating, and the number of times is counted.

[0098] (3) Check if the number of times predictions are made but updates are not made has reached the preset number. If it has, then the target will no longer be tracked and will be removed from the tracking list.

[0099] In this step, if the number of times the target is predicted but not updated during continuous tracking reaches a preset number, then the target will no longer be tracked and will be removed from the tracking list.

[0100] 6) If the feature similarity between the target obstacle and the tracked target is greater than a preset threshold, the matching is determined to be successful. Then, the motion state of the tracked target corresponding to the target obstacle at the previous moment is obtained from the matching relationship with the largest number of matches.

[0101] It should be noted that the matching relationship with the largest number of matches is determined through the following steps:

[0102] (1) If the target object matched by the target obstacle has been matched by other target obstacles, then the matching path is reversed to determine whether other target obstacles matched with the target object have a next matching target object;

[0103] (2) If so, match other target obstacles with the next matching tracking target, match the target obstacle with the tracking target, increment the matching number, and update the matching relationship;

[0104] (3) If not, check if the target obstacle has a next matching tracking target. If it does, match the target obstacle with the next matching tracking target and increment the matching number by one.

[0105] In this step, the motion state of the tracked target corresponding to the target obstacle at the previous moment is taken from the best matching relationship between the target obstacle and the tracked target. The best matching relationship is the matching relationship with the largest number of matches; for example, R S R represents the nodes (i.e., unmatched target obstacles) in the matching relation M that are not matched in the target obstacle list S. TTo track the nodes in list T that are not matched in matching relation M (i.e., the unmatched tracking targets), i.e., R S It includes nodes in S with an in-degree of 0, while R... T Includes the nodes in T with an out-degree of 0, and let Z be a node from R. S A directed path that follows only the tight edges can reach the tight-edge subgraph. The node, if R T If ∩Z is not empty, then the opposite is true. From R S Point to R T If the path is correct, increment the matching count by one; otherwise, move the new edge from S to T. Repeat the above steps, either by changing the value of potential y to increase it, or by changing the direction to get more edges, and keep all edges in the matching relation M as tight edges, until finally obtaining the perfect matching relation M with minimum cost.

[0106] S1032. Based on the motion state of the tracked target described in the previous moment, predict the motion state of the tracked target at the current moment to obtain the predicted value of the motion state at the current moment.

[0107] S1033. Correct the predicted value of the motion state at the current moment, determine the motion state of the current target obstacle, predict the motion state at the next moment based on the motion state at the current moment, and update the motion state of the tracked target in the tracking list from the previous moment to the motion state at the current moment.

[0108] For example, the target obstacle and the corresponding tracking target in the matching relationship are obtained by using the Hungarian algorithm. The next step is to predict and update the motion state of the tracking target. A robust extended Kalman filter can be used to perform motion estimation for all matched tracking targets in the tracking list to obtain their position and velocity.

[0109] It should be noted that the steps for correcting the predicted motion state of the current moment and determining the motion state of the target obstacle include:

[0110] 1) Obtain the observed values ​​of the target's motion state at the current moment from multiple sensors, correct the predicted value based on the observed values, and determine the corrected predicted value as the target's motion state at the current moment.

[0111] In this step, the velocity parameters of the robust extended Kalman filter are input by using observations from different sensors (such as anchor point displacement, displacement of the vertices and center of the bounding box, etc.) to prevent estimation bias caused by the failure of a single sensor through redundant observations; different position and velocity fusion strategies are designed based on historical information in the tracking list and the sensor type of the input observations.

[0112] It should be noted that the formula for predicting the current state based on the motion state at the previous moment is as follows:

[0113]

[0114]

[0115] in, Let g be the predicted value of the motion state at the current moment, and g be the formula for predicting the motion state. Let G be the covariance matrix. t Let ∑ be the state transition matrix. t-1 Let R be the covariance matrix of the previous time step. t This is the noise matrix.

[0116] The predicted value and covariance matrix are updated based on the current observation value, and then corrected. The corrected predicted value is determined as the motion state of the tracking target at the current moment. The specific formula is as follows:

[0117]

[0118]

[0119]

[0120] Among them, K t It is the Kalman gain, H is the observation matrix, and Q is the Kalman gain. t To observe the noise matrix, u t For the corrected motion state, z t For the observed values, ∑ t Let I be the updated covariance matrix, and let I be the identity matrix.

[0121] Here, the state transition equation for the constant velocity motion model is as follows:

[0122]

[0123] Where p represents position, v represents velocity, and a represents acceleration.

[0124] Here, after determining the corrected predicted value as the current motion state of the target being tracked, the path planning method further includes:

[0125] (1) Based on the point cloud data information of the tracked target, determine whether the tracked target is a tracked target after the fusion of lidar obstacle information and millimeter-wave radar obstacle information;

[0126] (2) If it is not fused, then obtain the obstacle information of the tracked target from the lidar scanning device and the millimeter-wave radar scanning device and fuse them.

[0127] In this step, after obtaining the motion state of the tracked target, it is determined whether it is a fused target obstacle. When there is no fused target obstacle, the obstacle information generated by the lidar and millimeter-wave radar is read, and the lidar and millimeter-wave radar are asynchronously fused to finally obtain the fused target obstacle.

[0128] S104. Based on the obstacle movement path, plan the driving path of the unmanned vehicle to avoid the obstacle.

[0129] For example, please refer to Figure 3 , Figure 3 This is a schematic diagram of the target tracking algorithm provided in the embodiments of this application, as shown below. Figure 3 As shown, during target tracking, to improve the robustness of the perception algorithm in rainy weather, an asynchronous fusion method of LiDAR and millimeter-wave radar is adopted. Millimeter-wave radar assists LiDAR to enhance the robustness of the target perception system. After obtaining accurate clustered targets, the asynchronous fusion of LiDAR and millimeter-wave radar accurately marks the targets to be tracked, achieving real-time target tracking. First, after clustering, LiDAR and millimeter-wave radar data can obtain a relatively dense target point cloud. Then, the Hungarian matching algorithm is used to perform more accurate frame-to-frame matching on these point clouds, providing good state information for motion estimation of target obstacles. Finally, based on the good state information, a constant velocity model is combined with robust Kalman filtering to improve the robust Kalman filtering method. Motion estimation is performed on all matched targets in the tracking list to obtain their position and velocity, thereby planning the driving path of the autonomous vehicle to avoid obstacles.

[0130] For example, the overall target tracking process includes: First, based on the target geometric information obtained from the LiDAR point cloud clustering, the Hungarian matching algorithm is used to match the LiDAR targets in consecutive frames. The selection criteria include: the target's center point, the target's velocity, and the target's length and width. Then, the LiDAR target tracking process is performed, mainly including:

[0131] 1. Based on the premise that the target undergoes uniformly accelerated motion, the target's state information at the current moment is estimated from the target's state information at the previous moment using the established state prediction matrix;

[0132] 2. The target matching relationship obtained through the Hungarian matching algorithm can provide the state information of the target object in the current frame, thereby enabling the updating of filtered measurements;

[0133] 3. For targets that are not matched by the Hungarian matching algorithm, if the target just appeared in the current frame, the state prediction process will be performed directly in the next frame. If the target existed in the previous frame but disappeared in this frame, the target will only be predicted in this frame and will not be updated. If the target only has a prediction process and no update process in the continuous process, the target will no longer be tracked.

[0134] Then, the target tracking data from the lidar and millimeter-wave radar are fused. Since millimeter-wave radar cannot accurately obtain the target's length and width information, the lidar data is used as the primary source in the fusion process, with the millimeter-wave radar serving as auxiliary information to improve target tracking accuracy and stability in rainy conditions. The principles of the fusion are as follows:

[0135] 1. When the prediction information is generated by lidar data, the next implementation waits for the target data from the lidar and millimeter-wave radar in the next frame. If it exists, the Hungarian matching algorithm is used to match the previous and next frames, and then the target state is predicted and updated. After that, robust Kalman filtering is used to estimate the motion of each target in the target tracking list.

[0136] 2. When the prediction information is generated by millimeter-wave radar data, the target tracking process in step 1 is also used while waiting for the target information of the next frame of lidar.

[0137] This application provides a path planning method, apparatus, electronic device, and storage medium for an autonomous vehicle. The path planning method includes: tracking obstacles based on obstacle information, predicting the motion state of the obstacles, and obtaining prediction information; wherein the obstacle information includes information acquired by at least one obstacle scanning device; the motion state includes speed information and distance information; fusing the obstacle information acquired by multiple devices based on the prediction information, and determining a target obstacle from the fused obstacle information; predicting the obstacle's motion path within a future predicted time period based on the target obstacle's motion state; and planning a driving path for the autonomous vehicle to avoid the obstacles based on the obstacle's motion path.

[0138] Thus, the technical solution provided in this application can obtain more accurate obstacles after fusion through predictive information, predict the movement path of the obstacle based on its movement state, and plan the obstacle avoidance path of the autonomous vehicle. This enables stable tracking of multiple targets in complex environments, improves the stability of the target tracking process while ensuring real-time performance, and allows for accurate obstacle avoidance path planning based on a stable target tracking process, thereby improving the accuracy of obstacle avoidance path planning for autonomous vehicles.

[0139] Based on the same application concept, this application also provides a path planning device for an autonomous vehicle, corresponding to the path planning method for an autonomous vehicle provided in the above embodiment. Since the principle of the device in this application is similar to the path planning method for an autonomous vehicle in the above embodiment, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0140] Please see Figure 4 , Figure 5 , Figure 4 This is one of the structural diagrams of a path planning device for an autonomous vehicle provided in an embodiment of this application. Figure 5 This is a second structural diagram of a path planning device for an autonomous vehicle provided in an embodiment of this application. Figure 4 As shown, the path planning device 410 includes:

[0141] The acquisition module 411 is used to track the obstacle based on obstacle information, predict the motion state of the obstacle, and acquire prediction information; wherein, the obstacle information includes information acquired by at least one obstacle scanning device; the motion state includes speed information and distance information;

[0142] The fusion module 412 is used to fuse the obstacle information obtained by multiple devices according to the prediction information, and determine the target obstacle from the fused obstacle information;

[0143] Prediction module 413 is used to predict the movement path of the target obstacle within a future prediction time period based on the movement state of the target obstacle;

[0144] The path planning module 414 is used to plan the driving path of the unmanned vehicle to avoid obstacles based on the obstacle movement path.

[0145] Optionally, when the fusion module 412 is used to determine the target obstacle, the fusion module 412 is specifically used for:

[0146] If the prediction information is generated by lidar, then wait to obtain the lidar obstacle information and millimeter-wave radar obstacle information of the obstacle at the next moment;

[0147] If the prediction information is generated by millimeter-wave radar, then wait to obtain the lidar obstacle information of the obstacle at the next moment;

[0148] Based on the obtained obstacle information, the target obstacle is identified.

[0149] Optional, such as Figure 5As shown, the path planning device 410 further includes a determination module 415, which is used for:

[0150] Based on the similarity between the characteristics of the target obstacle and each tracked target in the tracking list, the target obstacle is matched with each tracked target in the tracking list, and the motion state of the tracked target corresponding to the target obstacle under the matching relationship is obtained from the tracking list at the previous moment; wherein, the tracking list stores the motion state of each tracked target at the previous moment.

[0151] Based on the motion state of the tracked target described in the previous moment, the motion state of the tracked target at the current moment is predicted to obtain the predicted value of the motion state at the current moment.

[0152] The predicted value of the current motion state is corrected to determine the current motion state of the target obstacle. The motion state of the next moment is predicted based on the current motion state, and the previous motion state of the tracked target in the tracking list is updated to the current motion state.

[0153] Optionally, when the determining module 415 corrects the predicted value of the motion state at the current moment and determines the motion state of the current target obstacle, the determining module 415 is specifically used for:

[0154] The target's motion state at the current moment is observed by multiple sensors. The predicted value is then corrected based on the observed value, and the corrected predicted value is determined as the target's motion state at the current moment.

[0155] Optionally, when the determining module 415 is used to obtain the motion state of the tracking target corresponding to the target obstacle under the matching relationship at the previous moment, the determining module 415 is specifically used for:

[0156] Determine the correlation matrix between the list of target obstacles and the tracking list;

[0157] Based on the correlation matrix, obtain the corresponding bipartite graph;

[0158] The bipartite graph is divided into multiple subgraphs according to a preset splitting threshold.

[0159] For each subgraph, a matching process is performed between the target obstacle and the tracked target; wherein, the target obstacle is matched with at most one tracked target, and the tracked target is matched with at most one target obstacle.

[0160] Based on the feature similarity between the target obstacle and the tracked target, it is determined whether the match is successful;

[0161] If the feature similarity between the target obstacle and the tracked target is greater than a preset threshold, the match is determined to be successful. Then, the motion state of the tracked target corresponding to the target obstacle at the previous moment is obtained from the matching relationship with the largest number of matches.

[0162] Optionally, when determining the matching relationship with the largest number of matches, the determining module 415 is specifically used for:

[0163] Determine the match with the largest number of matches:

[0164] If the target object matched by the target obstacle has been matched by other target obstacles, then the matching path is reversed to determine whether other target obstacles matched with the target object have a next matching target object;

[0165] If present, match other target obstacles with the next matching tracking target, match the target obstacle with the tracking target, increment the matching count, and update the matching relationship;

[0166] If not, then check if the target obstacle has a next matching tracking target. If it does, then match the target obstacle with the next matching tracking target and increment the matching number by one.

[0167] Optionally, the determining module 415 is further configured to:

[0168] If the feature similarity between the target obstacle and the tracking target is less than a preset threshold, it is determined that the match is unsuccessful. Then, it is detected whether the unmatched target obstacle is appearing for the first time. If so, the target obstacle is added as a new tracking target to the tracking list, and the motion state of the target obstacle is predicted at the next moment.

[0169] If not, detect whether the tracked target has disappeared at the current moment. If the tracked target has disappeared at the current moment, predict the motion state of the tracked target, do not update the motion state, and increment the number of times only prediction is performed without updating.

[0170] If the number of times the prediction is made but not updated has reached a preset number, then the target will no longer be tracked and will be removed from the tracking list.

[0171] Optionally, the path planning device 410 further includes a judgment module 416, which is used for:

[0172] Based on the point cloud data information of the tracked target, determine whether the tracked target is a tracked target after the fusion of lidar obstacle information and millimeter-wave radar obstacle information;

[0173] If it is not a fused result, then obstacle information of the tracked target is obtained from the lidar scanning device and the millimeter-wave radar scanning device and fused.

[0174] This application provides a path planning device for an autonomous vehicle, comprising: an acquisition module for tracking obstacles based on obstacle information, predicting the movement state of the obstacles, and acquiring prediction information; wherein the obstacle information includes information acquired by at least one obstacle scanning device; and the movement state includes speed information and distance information; a fusion module for fusing the obstacle information acquired by multiple devices based on the prediction information, and determining a target obstacle from the fused obstacle information; a prediction module for predicting the obstacle movement path of the target obstacle within a future predicted time period based on the movement state of the target obstacle; and a path planning module for planning a driving path for the autonomous vehicle to avoid obstacles based on the obstacle movement path.

[0175] Thus, the technical solution provided in this application can obtain more accurate obstacles after fusion through predictive information, predict the movement path of the obstacle based on its movement state, and plan the obstacle avoidance path of the autonomous vehicle. This enables stable tracking of multiple targets in complex environments, improves the stability of the target tracking process while ensuring real-time performance, and allows for accurate obstacle avoidance path planning based on a stable target tracking process, thereby improving the accuracy of obstacle avoidance path planning for autonomous vehicles.

[0176] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 600 includes a processor 610, a memory 620, and a bus 630.

[0177] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 and the memory 620 communicate via the bus 630. When the machine-readable instructions are executed by the processor 610, they can perform the operations described above. Figure 1 as well as Figure 2 The steps of the path planning method for the autonomous vehicle in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0178] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 as well as Figure 2The steps of the path planning method for the autonomous vehicle in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0179] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0183] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0184] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A path planning method for an autonomous vehicle, characterized in that, The autonomous vehicle is equipped with various obstacle scanning devices, and the path planning method includes: The obstacle is tracked based on obstacle information, and the motion state of the obstacle is predicted to obtain prediction information; wherein, the obstacle information includes information obtained by at least one obstacle scanning device; the motion state includes speed information and distance information; Based on the predicted information, the obstacle information obtained by multiple devices is fused, and the target obstacle is determined from the fused obstacle information; Based on the motion state of the target obstacle, the obstacle's motion path within a future predicted time period is predicted; Based on the obstacle's movement path, plan the autonomous vehicle's driving path to avoid the obstacle; The motion state of the target obstacle is determined by the following steps: Based on the similarity between the characteristics of the target obstacle and each tracked target in the tracking list, the target obstacle is matched with each tracked target in the tracking list, and the motion state of the tracked target corresponding to the target obstacle under the matching relationship is obtained from the tracking list at the previous moment; wherein, the tracking list stores the motion state of each tracked target at the previous moment. Based on the motion state of the tracked target described in the previous moment, the motion state of the tracked target at the current moment is predicted to obtain the predicted value of the motion state at the current moment. The predicted value of the current motion state is corrected to determine the current motion state of the target obstacle. The motion state of the next moment is predicted based on the current motion state, and the previous motion state of the tracked target in the tracking list is updated to the current motion state. The following steps are used to obtain the motion state of the tracked target corresponding to the target obstacle under the matching relationship at the previous moment: Determine the correlation matrix between the list of target obstacles and the tracking list; Based on the correlation matrix, obtain the corresponding bipartite graph; The bipartite graph is divided into multiple subgraphs according to a preset splitting threshold. For each subgraph, a matching process is performed between the target obstacle and the tracked target; wherein, the target obstacle is matched with at most one tracked target, and the tracked target is matched with at most one target obstacle. Based on the feature similarity between the target obstacle and the tracked target, it is determined whether the match is successful; If the feature similarity between the target obstacle and the tracked target is greater than a preset threshold, the match is determined to be successful. Then, the motion state of the tracked target corresponding to the target obstacle at the previous moment is obtained from the matching relationship with the largest number of matches. The following steps are used to determine the matching relationship with the largest number of matches: If the target object matched by the target obstacle has been matched by other target obstacles, then the matching path is reversed to determine whether other target obstacles matched with the target object have a next matching target object; If present, match other target obstacles with the next matching tracking target, match the target obstacle with the tracking target, increment the matching count, and update the matching relationship; If not, then check if the target obstacle has a next matching tracking target. If it does, then match the target obstacle with the next matching tracking target and increment the matching number by one. After determining whether a match is successful, the path planning method further includes: If the feature similarity between the target obstacle and the tracking target is less than a preset threshold, it is determined that the match is unsuccessful. Then, it is detected whether the unmatched target obstacle is appearing for the first time. If so, the target obstacle is added as a new tracking target to the tracking list, and the motion state of the target obstacle is predicted at the next moment. If not, detect whether the tracked target has disappeared at the current moment. If the tracked target has disappeared at the current moment, predict the motion state of the tracked target, do not update the motion state, and increment the number of times only prediction is performed without updating. If the number of times the prediction is made but not updated has reached a preset number, then the target will no longer be tracked and will be removed from the tracking list.

2. The path planning method according to claim 1, characterized in that, The target obstacle is identified through the following steps: If the prediction information is generated by lidar, then wait to obtain the lidar obstacle information and millimeter-wave radar obstacle information of the obstacle at the next moment; If the prediction information is generated by millimeter-wave radar, then wait to obtain the lidar obstacle information of the obstacle at the next moment; Based on the obtained obstacle information, the target obstacle is identified.

3. The path planning method according to claim 1, characterized in that, The step of correcting the predicted value of the motion state at the current moment to determine the motion state of the target obstacle includes: The target's motion state at the current moment is observed by multiple sensors. The predicted value is then corrected based on the observed value, and the corrected predicted value is determined as the target's motion state at the current moment.

4. The path planning method according to claim 3, characterized in that, After determining the corrected predicted value as the current motion state of the tracking target, the path planning method further includes: Based on the point cloud data information of the tracked target, determine whether the tracked target is a tracked target after the fusion of lidar obstacle information and millimeter-wave radar obstacle information; If it is not a fused result, then obstacle information of the tracked target is obtained from the lidar scanning device and the millimeter-wave radar scanning device and fused.

5. A path planning device for an autonomous vehicle, characterized in that, The path planning device includes: The acquisition module is used to track the obstacle based on obstacle information, predict the motion state of the obstacle, and acquire prediction information; wherein, the obstacle information includes information acquired by at least one obstacle scanning device; the motion state includes speed information and distance information; The fusion module is used to fuse the obstacle information obtained by multiple devices based on the prediction information, and determine the target obstacle from the fused obstacle information; The prediction module is used to predict the movement path of the target obstacle within a future predicted time period based on the movement state of the target obstacle. The path planning module is used to plan the driving path of the autonomous vehicle to avoid obstacles based on the movement path of the obstacles. The path planning device further includes a determination module, which is used for: Based on the similarity between the characteristics of the target obstacle and each tracked target in the tracking list, the target obstacle is matched with each tracked target in the tracking list, and the motion state of the tracked target corresponding to the target obstacle under the matching relationship is obtained from the tracking list at the previous moment; wherein, the tracking list stores the motion state of each tracked target at the previous moment. Based on the motion state of the tracked target described in the previous moment, the motion state of the tracked target at the current moment is predicted to obtain the predicted value of the motion state at the current moment. The predicted value of the current motion state is corrected to determine the current motion state of the target obstacle. The motion state of the next moment is predicted based on the current motion state, and the previous motion state of the tracked target in the tracking list is updated to the current motion state. When the determining module is used to obtain the motion state of the tracking target corresponding to the target obstacle under the matching relationship at the previous moment, the determining module is specifically used for: Determine the correlation matrix between the list of target obstacles and the tracking list; Based on the correlation matrix, obtain the corresponding bipartite graph; The bipartite graph is divided into multiple subgraphs according to a preset splitting threshold. For each subgraph, a matching process is performed between the target obstacle and the tracked target; wherein, the target obstacle is matched with at most one tracked target, and the tracked target is matched with at most one target obstacle. Based on the feature similarity between the target obstacle and the tracked target, it is determined whether the match is successful; If the feature similarity between the target obstacle and the tracked target is greater than a preset threshold, the match is determined to be successful. Then, the motion state of the tracked target corresponding to the target obstacle at the previous moment is obtained from the matching relationship with the largest number of matches. When determining the matching relationship with the largest number of matches, the determining module is specifically used for: Determine the match with the largest number of matches: If the target object matched by the target obstacle has been matched by other target obstacles, then the matching path is reversed to determine whether other target obstacles matched with the target object have a next matching target object; If present, match other target obstacles with the next matching tracking target, match the target obstacle with the tracking target, increment the matching count, and update the matching relationship; If not, then check if the target obstacle has a next matching tracking target. If it does, then match the target obstacle with the next matching tracking target and increment the matching number by one. The determining module is also used for: If the feature similarity between the target obstacle and the tracking target is less than a preset threshold, it is determined that the match is unsuccessful. Then, it is detected whether the unmatched target obstacle is appearing for the first time. If so, the target obstacle is added as a new tracking target to the tracking list, and the motion state of the target obstacle is predicted at the next moment. If not, detect whether the tracked target has disappeared at the current moment. If the tracked target has disappeared at the current moment, predict the motion state of the tracked target, do not update the motion state, and increment the number of times only prediction is performed without updating. If the number of times the prediction is made but not updated has reached a preset number, then the target will no longer be tracked and will be removed from the tracking list.

6. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the path planning method for an autonomous vehicle as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the path planning method for an autonomous vehicle as described in any one of claims 1 to 4.

Images