Vehicle trajectory prediction method, device, electronic device and storage medium

Abstract

The present invention provides a vehicle trajectory prediction method, device, electronic device and storage medium. The method comprises: obtaining at least one first candidate trajectory curve of a target vehicle by vision; wherein the first candidate trajectory curve comprises a lane line curve, a curb curve and a drivable area boundary curve; obtaining a second candidate trajectory curve of the target vehicle by radar; wherein the second candidate curve comprises a road marker curve; selecting the road trajectory of the road where the target vehicle is located from each of the first candidate trajectory curve and the second candidate trajectory curve, and using it as the first predicted trajectory; determining the target predicted trajectory of the target vehicle based on the first predicted trajectory. Compared with the trajectory prediction of a single sensor, the present invention has better stability and higher accuracy.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a vehicle trajectory prediction method, device, electronic equipment and storage medium. Background Art

[0002] In the field of intelligent driving, predicting the vehicle's driving trajectory is crucial. Typical methods for predicting vehicle trajectory include using the vehicle posture module to collect the vehicle's three-axis angular velocity and three-axis acceleration to calculate the vehicle's posture and driving trajectory, or relying on the steering wheel angle and the vehicle's transmission ratio coefficient to calculate the driving trajectory based on the vehicle model.

[0003] All of the above use sensors to measure the state parameters of the vehicle to predict the vehicle's driving trajectory. However, the measurement values ​​of the sensors are discrete, and the vehicle trajectory prediction results obtained based on the measurement values ​​are unstable and have certain inaccuracies. Summary of the invention

[0004] Embodiments of the present invention provide a vehicle trajectory prediction method, device, electronic device and storage medium to solve the problem of improving the accuracy of vehicle trajectory prediction.

[0005] In a first aspect, an embodiment of the present invention provides a vehicle trajectory prediction method, comprising:

[0006] Acquire at least one first candidate trajectory curve of the target vehicle through vision; wherein the first candidate trajectory curve includes a lane line curve, a curb curve, and a drivable area boundary curve;

[0007] Acquire a second candidate trajectory curve of the target vehicle by radar; wherein the second candidate curve includes a road marker curve;

[0008] Selecting a road trajectory of the road where the target vehicle is located from each of the first candidate trajectory curves and the second candidate trajectory curves as a first predicted trajectory;

[0009] A target predicted trajectory of the target vehicle is determined based on the first predicted trajectory.

[0010] In a possible implementation, selecting a road trajectory of a road where the target vehicle is located from each of the first candidate trajectory curves and the second candidate trajectory curves includes:

[0011] Calculate the similarity between each first candidate trajectory curve and the second candidate trajectory curve and the previous frame prediction trajectory;

[0012] Based on the confidence and similarity of each of the first candidate trajectory curves and the second candidate trajectory curves, a road trajectory of the road where the target vehicle is located is selected from each of the candidate trajectory curves.

[0013] In a possible implementation, obtaining a second candidate trajectory curve of the target vehicle by radar includes:

[0014] Obtaining position information of multiple stationary targets detected by the radar of the target vehicle;

[0015] Based on the distances between the stationary targets, the stationary targets are divided into a plurality of groups of stationary targets;

[0016] Performing curve fitting on the position information of each group of stationary targets respectively to obtain a plurality of initial marker curves, and obtaining the confidence of each initial marker curve;

[0017] Each initial marker curve is screened based on the confidence level to obtain a preset number of road marker curves.

[0018] In a possible implementation, the stationary targets are divided into multiple groups of stationary targets based on the distances between the stationary targets, including:

[0019] Multiple stationary targets whose lateral distance between two adjacent stationary targets is less than a first threshold, whose longitudinal distance between two adjacent stationary targets is less than a second threshold, and whose maximum distance among all stationary targets is greater than a third threshold are divided into a group of stationary targets; wherein the first threshold is less than the second threshold, and the second threshold is less than the third threshold.

[0020] In a possible implementation, determining a target predicted trajectory of the target vehicle based on the first predicted trajectory includes:

[0021] If there is no lane line curve with a confidence level higher than a preset first confidence level in each of the first candidate trajectory curves, and there is no road marker curve with a confidence level higher than a preset second confidence level in each of the second candidate trajectory curves, then obtaining a second predicted trajectory corresponding to the vehicle body posture of the target vehicle, and / or obtaining a third predicted trajectory corresponding to the steering wheel angle of the target vehicle;

[0022] The first predicted trajectory, the second predicted trajectory and / or the third predicted trajectory are fused to obtain a target predicted trajectory of the target vehicle.

[0023] In a possible implementation, the first predicted trajectory includes a first left predicted trajectory and a first right predicted trajectory; the first predicted trajectory, the second predicted trajectory and / or the third predicted trajectory are merged to obtain a target predicted trajectory of the target vehicle, including:

[0024] Move the first left predicted trajectory, and the starting point of the second predicted trajectory and / or the third predicted trajectory to the left starting point (x1,0) of the target vehicle in the trajectory coordinate system;

[0025] Taking points on the first left prediction track, the second prediction track and / or the third prediction track according to a preset y-axis interval to obtain a plurality of left prediction points;

[0026] Fit each left prediction point to obtain the left prediction trajectory;

[0027] and moving the first right predicted trajectory, and the starting point of the second predicted trajectory and / or the third predicted trajectory to the right starting point (x2,0) of the target vehicle in the trajectory coordinate system;

[0028] Taking points on the first right prediction track, the second prediction track and / or the third prediction track according to a preset y-axis interval to obtain a plurality of right prediction points;

[0029] Fit each right-side prediction point to obtain the right-side prediction trajectory;

[0030] The left predicted trajectory and the right predicted trajectory are forced to be parallel to obtain the target predicted trajectory of the target vehicle.

[0031] In a second aspect, an embodiment of the present invention provides a vehicle trajectory prediction device, comprising:

[0032] A first acquisition module, used to acquire at least one first candidate trajectory curve of the target vehicle by vision; wherein the first candidate trajectory curve includes a lane line curve, a curb curve and a drivable area boundary curve;

[0033] A second acquisition module, used to acquire a second candidate trajectory curve of the target vehicle through a radar; wherein the second candidate curve includes a road marker curve;

[0034] A selection module, used to select a road trajectory of the road where the target vehicle is located from each of the first candidate trajectory curves and the second candidate trajectory curves, and use the road trajectory as the first predicted trajectory;

[0035] The prediction module is used to determine a target predicted trajectory of the target vehicle based on the first predicted trajectory.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements the steps of the method described in the first aspect or any possible implementation manner of the first aspect.

[0037] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation method of the first aspect are implemented.

[0038] The embodiments of the present invention provide a vehicle trajectory prediction method, device, electronic device and storage medium. The method detects regular stationary markers on the road where the target vehicle is located by radar, and uses the characteristic that the curve formed by connecting the markers is parallel to the road direction to determine the road marker curve close to the actual trajectory of the target vehicle. Then, a suitable curve is selected from the road marker curve, lane line curve, curb curve and drivable area boundary curve as the first predicted trajectory, thereby realizing the fusion of multiple parameters related to the vehicle's driving trajectory. Compared with the trajectory prediction of a single sensor, the method has better stability and higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 This is an application scenario diagram of a vehicle trajectory prediction method provided by an embodiment of the present invention;

[0041] Figure 2 is a flow chart of an implementation of a vehicle trajectory prediction method provided by an embodiment of the present invention;

[0042] Figure 3A is an original image captured by a camera provided in an embodiment of the present invention;

[0043] Figure 3B is a target detection result diagram provided by an embodiment of the present invention;

[0044] Figure 3C is a target segmentation schematic diagram provided by an embodiment of the present invention;

[0045] Figure 3D is a schematic diagram of a detection target provided by an embodiment of the present invention;

[0046] Figure 4 is a comparison diagram of candidate trajectory curves provided by an embodiment of the present invention;

[0047] Figure 5 is a flow chart of an implementation of a vehicle trajectory prediction method provided by another embodiment of the present invention;

[0048] Figure 6 is a schematic diagram of the structure of a vehicle trajectory prediction device provided by an embodiment of the present invention;

[0049] Figure 7 It is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0052] Figure 1 FIG. 1 is an application scenario diagram of the vehicle trajectory prediction method provided by an embodiment of the present invention. Figure 1 As shown in the figure, the basic components of the vehicle system include: domain controller, vehicle posture acquisition module, GPS module, decision and execution module, radar, and camera. Among them, the domain controller mainly receives signals and processes, fuses, stores data, and makes decisions on the received data. The global positioning module is used to collect the location of the vehicle. The radar is generally a millimeter-wave radar. The Advanced Driving Assistance System (ADAS) camera is a visual sensor used to capture images of the viewing angle. The cloud acts as a server and the domain controller to transmit signals, videos and other data through the 4G / 5G module. The Driver State Monitoring (DMS) camera can detect the driver's behavior and identify the driving status, such as distracted driving, making phone calls, fatigue, closing eyes, etc.

[0053] The overall workflow of the system is that the radar outputs the target list signal, the camera outputs the image, the domain controller receives the image and performs target recognition, lane line detection, drivable area detection, etc. on the image. After detecting and recognizing the image target, the target list and lane line information can be obtained through post-processing. The visual target list obtained by the image is integrated with the radar target list obtained by the radar, and combined with the vehicle's posture, speed and other signals, functions such as forward automatic emergency braking (Autonomous Emergency Braking-Vehicle, AEB), adaptive cruise control (Adaptive Cruise Control, ACC), and lane keeping assist (Lane Keep Assist, LKA) can be completed.

[0054] Generally, the prediction of driving trajectory is usually done by using physical perception sensors such as six-axis body posture sensors and steering wheel angles combined with information such as vehicle speed. The above methods all use sensors to directly detect the driving state of the vehicle and then use mathematical geometric calculations to determine it. However, sensor detection will have different degrees of interference under different working conditions. For example, the six-axis sensor will be affected by vehicle acceleration and deceleration, and by road bumps, resulting in unstable and inaccurate trajectory prediction.

[0055] See also Figure 2 , which shows a flow chart of the implementation of the vehicle trajectory prediction method provided by an embodiment of the present invention, which is described in detail as follows:

[0056] Step 201, obtaining at least one first candidate trajectory curve of the target vehicle by vision; wherein the first candidate trajectory curve includes a lane line curve, a curb curve and a drivable area boundary curve.

[0057] In this embodiment, when the camera is connected to the system, the domain controller collects the camera image and uses deep learning methods to perform target detection, lane line detection and image segmentation, which can segment the image to obtain elements such as the drivable area, curb, vehicle, pedestrian, etc. Then, the camera and radar can be used to analyze and calculate the road contour and predict the vehicle's driving trajectory.

[0058] After obtaining multiple first candidate trajectory curves related to the road contour, the lane line, curb, drivable area, and radar information can be fused together or partially fused to determine the road contour. Different strategies can be applied to achieve the fusion of multiple curves for different scenarios. For example, when certain types of curves cannot be obtained or are too low in confidence to be used, all available curves will be fused. The following is a step-by-step introduction to the specific method of obtaining the first candidate trajectory curve.

[0059] Lane information: Lanes can come from segmentation detection or lane detection. The processing flow is to extract the outline of the lane type -> cluster the same lane according to pixel distance -> remove invalid points and outliers -> image coordinate linear fitting -> world coordinate system projection -> fitting in the world coordinate system -> linear coefficient filtering -> output curve.

[0060] Curb information: The extraction process of curb information is similar to that of lane lines. The process is as follows: extract the outline of the curb -> classify the curb into blocks -> filter out the left and right curbs -> image-side fitting -> world coordinate system projection -> world coordinate system fitting -> filtering -> fitting.

[0061] Estimation of road boundary information in the drivable area: mainly involves the extraction of segmentation, and the process is as follows: extract the boundary of the drivable area to form a closed area -> take the line segments of the leftmost and rightmost boundary lines (interrupted when encountering other categories) -> associate and classify the line segments to form two left and right boundary lines -> image end fitting -> world coordinate system projection -> world coordinate system fitting -> coefficient filtering -> output the drivable area boundary curve.

[0062] Step 202, obtaining a second candidate trajectory curve of the target vehicle through radar; wherein the second candidate curve includes a road marker curve.

[0063] In this embodiment, the radar detects the target on the road where the vehicle is located, and can output a parameter list of the target, including the target's lateral distance, longitudinal distance, lateral speed, longitudinal speed, acceleration and other information. Road markers can be objects such as trees, street lights, guardrails and the like on both sides of the road. The positions of such objects have regularity. The distance between each road marker and the edge of the road is basically the same among the road markers of the same category, so the curve formed by connecting the positions of the road markers of the same category is also basically the same as the road profile.

[0064] By using various parameters of the detected targets, targets corresponding to road markers can be screened out, and road marker curves can be fitted to determine the road profile.

[0065] Step 203 : Selecting a road trajectory of the road where the target vehicle is located from each of the first candidate trajectory curves and the second candidate trajectory curves as the first predicted trajectory.

[0066] In this embodiment, Figure 3A It is the original image captured by the camera. After processing, this image is used to pass into the neural network and detect various objects of interest. Figure 3B This is the target detection result diagram, in which target detection and lane line detection are performed respectively. Figure 3C It is a segmentation map, which mainly completes the segmentation of drivable areas, lane lines, curbs, vehicles, pedestrians and other targets. Figure 3D It is a bird's-eye view of the target, which mainly includes radar targets (dots) and visual targets (horizontal lines).

[0067] Figure 4 Here, 0 represents the predicted trajectory of the target vehicle, 1 represents the lane line, 2 represents the curb, 3 represents the boundary of the drivable area, and 4 represents the road marker curve based on radar detection and fitting.

[0068] When the target vehicle does not change lanes, the vehicle's driving trajectory follows the road trend. Therefore, in this embodiment, the most suitable curve is selected from multiple candidate prediction curves that can reflect the road contour to represent the road trajectory and serve as the basis for determining the target prediction trajectory of the target vehicle.

[0069] Step 204 : determining a target predicted trajectory of the target vehicle based on the first predicted trajectory.

[0070] In this embodiment, in order to ensure the accuracy of the target prediction trajectory, it can be set that the first prediction trajectory can be directly used as the target prediction trajectory only when there are lane lines or road marker curves. When there are no available lane lines and road marker curves, it is difficult to determine the driving direction of the vehicle only through the curb information and the boundary of the drivable area. At this time, the selected first prediction trajectory cannot be used directly, and the parameters of other prediction trajectories need to be introduced for fusion to determine the accurate target prediction trajectory.

[0071] The embodiment of the present invention detects regular stationary markers on the road where the target vehicle is located through radar, and utilizes the characteristic that the curve formed by connecting the markers is parallel to the direction of the road to determine the road marker curve close to the actual trajectory of the target vehicle, and then selects a suitable curve from the road marker curve, lane line curve, curb curve and drivable area boundary curve as the first predicted trajectory, thereby realizing the fusion of multiple parameters related to the vehicle's driving trajectory, and having better stability and higher accuracy than the trajectory prediction of a single sensor.

[0072] In a possible implementation, selecting a road trajectory of a road where the target vehicle is located from each of the first candidate trajectory curves and the second candidate trajectory curves includes:

[0073] Calculate the similarity between each first candidate trajectory curve and the second candidate trajectory curve and the previous frame prediction trajectory;

[0074] Based on the confidence and similarity of each of the first candidate trajectory curves and the second candidate trajectory curves, a road trajectory of the road where the target vehicle is located is selected from each of the candidate trajectory curves.

[0075] In this embodiment, the predicted trajectory output and displayed to the user is continuously updated. Taking into account the user experience, the predicted trajectories of two adjacent frames should be as close as possible. In actual use, it is almost impossible for the driving trajectory of the target vehicle to change suddenly. Therefore, the similarity between each first candidate trajectory curve and the second candidate trajectory curve and the predicted trajectory of the previous frame can be calculated, and the curve with higher similarity can be selected as the road trajectory of the road where the target vehicle is located.

[0076] The higher the confidence of the curve, the more authentic the curve is, so the curve with higher confidence can also be selected as the road trajectory of the road where the target vehicle is located. In this embodiment, the confidence and similarity of the curve can also be considered at the same time, for example, the weights of confidence and similarity are set according to actual needs to select the curve.

[0077] When determining the predicted trajectory of the first frame, since there is no predicted trajectory of the previous frame, the curve selection can be performed only based on the confidence level, or the user can select the road trajectory of the road where the target vehicle is located from multiple candidate trajectory curves.

[0078] In addition, after a period of trajectory prediction, the actual trajectory of the target vehicle can be recorded, and the predicted trajectory of the previous frame can be corrected, and the similarity of the curve can be calculated based on the corrected predicted trajectory.

[0079] In a possible implementation, obtaining a second candidate trajectory curve of the target vehicle by radar includes:

[0080] Obtaining position information of multiple stationary targets detected by the radar of the target vehicle;

[0081] Based on the distances between the stationary targets, the stationary targets are divided into a plurality of groups of stationary targets;

[0082] Performing curve fitting on the position information of each group of stationary targets respectively to obtain a plurality of initial marker curves, and obtaining the confidence of each initial marker curve;

[0083] Each initial marker curve is screened based on the confidence level to obtain a preset number of road marker curves.

[0084] In this embodiment, since there is a regularity in the positions of markers of the same category, the markers of the same category can be divided into one group using the distance between stationary targets for curve fitting.

[0085] The confidence of the initial marker curve can be determined based on the confidence of the radar data at each point in the curve, for example, the average of the confidence of the radar data at each point in the curve is used as the confidence of the curve.

[0086] In a possible implementation, the stationary targets are divided into multiple groups of stationary targets based on the distances between the stationary targets, including:

[0087] Multiple stationary targets whose lateral distance between two adjacent stationary targets is less than a first threshold, whose longitudinal distance between two adjacent stationary targets is less than a second threshold, and whose maximum distance among all stationary targets is greater than a third threshold are divided into a group of stationary targets; wherein the first threshold is less than the second threshold, and the second threshold is less than the third threshold.

[0088] In this embodiment, under regular road conditions, there will be regular roadside reflective object information in most cases. The overall process of extracting effective information from radar detection targets can be as follows:

[0089] Filter stationary radar targets (because road guardrails, trees, etc. are stationary targets) -> Cluster based on the distance of radar points -> Verify the clustering results, filter qualified clusters for merge classification (such as the total point distance coverage is greater than 40m, the lateral distance between adjacent points is less than 1.5m, the longitudinal distance between adjacent points is less than 5m, etc.) -> Filter up to two clusters with higher confidence, corresponding to the left and right sides of the road respectively -> Perform least squares fitting with the coordinate points of each target -> Output the road line predicted by radar.

[0090] In a possible implementation, determining a target predicted trajectory of the target vehicle based on the first predicted trajectory includes:

[0091] If there is no lane line curve with a confidence level higher than a preset first confidence level in each of the first candidate trajectory curves, and there is no road marker curve with a confidence level higher than a preset second confidence level in each of the second candidate trajectory curves, then obtaining a second predicted trajectory corresponding to the vehicle body posture of the target vehicle, and / or obtaining a third predicted trajectory corresponding to the steering wheel angle of the target vehicle;

[0092] The first predicted trajectory, the second predicted trajectory and / or the third predicted trajectory are fused to obtain a target predicted trajectory of the target vehicle.

[0093] In this embodiment, the calculation steps for obtaining the second predicted trajectory corresponding to the body posture (yawrate) of the target vehicle are:

[0094] The vehicle's curvature radius is calculated based on the yawrate, R=V / w, where R is the vehicle's curvature radius, V is the vehicle's speed in the direction of travel, and w is the angular velocity of rotation. The vehicle's curvature radius is the vehicle's driving trajectory estimated by the inertial sensor.

[0095] The specific method of obtaining the third predicted trajectory corresponding to the steering wheel angle of the target vehicle is:

[0096] The vehicle's driving trajectory is obtained according to the steering wheel angle, and the vehicle motion model is simplified according to the Ackerman geometry relationship, so that the vehicle's driving radius can be calculated. The predicted driving path obtained according to the steering wheel angle is not directly affected by the bumps of the road.

[0097]

[0098]

[0099] in, is the steering wheel angle, is the wheelbase of the vehicle, is the lateral position of the vehicle, is the lateral acceleration (centripetal acceleration) of the vehicle, is the speed of the vehicle, is the steering system transmission ratio, is the turning radius of the vehicle.

[0100] In a possible implementation, the first predicted trajectory includes a first left predicted trajectory and a first right predicted trajectory; the first predicted trajectory, the second predicted trajectory and / or the third predicted trajectory are merged to obtain a target predicted trajectory of the target vehicle, including:

[0101] Move the first left predicted trajectory, and the starting point of the second predicted trajectory and / or the third predicted trajectory to the left starting point (x1,0) of the target vehicle in the trajectory coordinate system;

[0102] Taking points on the first left prediction track, the second prediction track and / or the third prediction track according to a preset y-axis interval to obtain a plurality of left prediction points;

[0103] Fit each left prediction point to obtain the left prediction trajectory;

[0104] and moving the first right predicted trajectory, and the starting point of the second predicted trajectory and / or the third predicted trajectory to the right starting point (x2,0) of the target vehicle in the trajectory coordinate system;

[0105] Taking points on the first right prediction track, the second prediction track and / or the third prediction track according to a preset y-axis interval to obtain a plurality of right prediction points;

[0106] Fit each right-side prediction point to obtain the right-side prediction trajectory;

[0107] The left predicted trajectory and the right predicted trajectory are forced to be parallel to obtain the target predicted trajectory of the target vehicle.

[0108] In this embodiment, taking the fusion of three curve equations as an example, the point equation of vision and radar is y=f1(x), the equation obtained by yawrate is y=f2(x), and the equation obtained by steering wheel angle is y=f3(x). First, the starting points of the three curves are translated to the left starting point (x1, 0) and the right starting point (x2, 0) of the target vehicle according to the left and right sides, and then the points are obtained with a spacing of 1m on the y axis. The method of obtaining the points can be to average the x coordinates of the points on the three curves, and the point coordinates are obtained as ( ), get the points corresponding to the three curves, then put the left points together, put the right points together, and perform quadratic fitting on the left and right curves respectively, so as to ensure the robustness of the data. Finally, force the curves on both sides to be parallel and output.

[0109] To ensure the fusion effect, points can also be obtained based on the confidence of each curve. Specifically, the confidence of each point in the curve can be used, with 1m as the y-axis spacing, and the confidence of the point corresponding to the coordinate on each curve can be judged on the y-axis coordinate of each obtained point, and the point with the highest confidence can be selected as the obtained point.

[0110] The confidence of the entire curve can also be used to select the curve with the highest confidence from the left curve and the right curve as the fused curve, and discard the curve with low confidence to avoid the curve with low confidence from interfering with the fusion result.

[0111] In a specific embodiment, see Figure 5 ,The trajectory prediction of the target vehicle is divided into two parallel parts, Figure 5 On the left are the steps for determining the road contour of the target vehicle. The image captured by the camera is subjected to target detection, image segmentation, lane line extraction, curb information extraction and drivable area boundary extraction. The coordinates of the target detected by the radar are fitted to achieve road marker curve fitting. Filtering is performed in each step to ensure the accuracy of the extracted data. Then, based on the confidence level and the similarity with the predicted trajectory of the previous frame, the first predicted trajectory corresponding to the road contour is selected. Figure 5 The right side shows the steps of obtaining the second predicted trajectory corresponding to the body posture of the target vehicle and obtaining the third predicted trajectory corresponding to the steering wheel angle of the target vehicle. Finally, the first predicted trajectory, the second predicted trajectory and the third predicted trajectory are fused to obtain the final target predicted trajectory.

[0112] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0113] The following is an embodiment of the device of the present invention. For details not described in detail, reference may be made to the corresponding method embodiments described above.

[0114] Figure 6 The structure diagram of the vehicle trajectory prediction device provided by the embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0115] like Figure 6 As shown, the vehicle trajectory prediction device 6 includes:

[0116] A first acquisition module 61 is used to obtain at least one first candidate trajectory curve of the target vehicle by vision; wherein the first candidate trajectory curve includes a lane line curve, a curb curve and a drivable area boundary curve;

[0117] A second acquisition module 62 is used to acquire a second candidate trajectory curve of the target vehicle through a radar; wherein the second candidate curve includes a road marker curve;

[0118] A selection module 63, configured to select a road trajectory of a road where a target vehicle is located from each of the first candidate trajectory curves and the second candidate trajectory curves, and use the road trajectory as a first predicted trajectory;

[0119] The prediction module 64 is configured to determine a target predicted trajectory of the target vehicle based on the first predicted trajectory.

[0120] In a possible implementation, the selection module 63 is specifically used for:

[0121] Calculate the similarity between each first candidate trajectory curve and the second candidate trajectory curve and the previous frame prediction trajectory;

[0122] Based on the confidence and similarity of each of the first candidate trajectory curves and the second candidate trajectory curves, a road trajectory of the road where the target vehicle is located is selected from each of the candidate trajectory curves.

[0123] In a possible implementation, the second acquisition module 62 is specifically configured to:

[0124] Obtaining position information of multiple stationary targets detected by the radar of the target vehicle;

[0125] Based on the distances between the stationary targets, the stationary targets are divided into a plurality of groups of stationary targets;

[0126] Performing curve fitting on the position information of each group of stationary targets respectively to obtain a plurality of initial marker curves, and obtaining the confidence of each initial marker curve;

[0127] Each initial marker curve is screened based on the confidence level to obtain a preset number of road marker curves.

[0128] In a possible implementation, the second acquisition module 62 is specifically configured to:

[0129] Multiple stationary targets whose lateral distance between two adjacent stationary targets is less than a first threshold, whose longitudinal distance between two adjacent stationary targets is less than a second threshold, and whose maximum distance among all stationary targets is greater than a third threshold are divided into a group of stationary targets; wherein the first threshold is less than the second threshold, and the second threshold is less than the third threshold.

[0130] In a possible implementation, the prediction module 64 is specifically used for:

[0131] If there is no lane line curve with a confidence level higher than a preset first confidence level in each of the first candidate trajectory curves, and there is no road marker curve with a confidence level higher than a preset second confidence level in each of the second candidate trajectory curves, then obtaining a second predicted trajectory corresponding to the vehicle body posture of the target vehicle, and / or obtaining a third predicted trajectory corresponding to the steering wheel angle of the target vehicle;

[0132] The first predicted trajectory, the second predicted trajectory and / or the third predicted trajectory are fused to obtain a target predicted trajectory of the target vehicle.

[0133] In a possible implementation, the first predicted trajectory includes a first left predicted trajectory and a first right predicted trajectory; the prediction module 64 is specifically configured to:

[0134] Move the first left predicted trajectory, and the starting point of the second predicted trajectory and / or the third predicted trajectory to the left starting point (x1,0) of the target vehicle in the trajectory coordinate system;

[0135] Taking points on the first left prediction track, the second prediction track and / or the third prediction track according to a preset y-axis interval to obtain a plurality of left prediction points;

[0136] Fit each left prediction point to obtain the left prediction trajectory;

[0137] and moving the first right predicted trajectory, and the starting point of the second predicted trajectory and / or the third predicted trajectory to the right starting point (x2,0) of the target vehicle in the trajectory coordinate system;

[0138] Taking points on the first right prediction track, the second prediction track and / or the third prediction track according to a preset y-axis interval to obtain a plurality of right prediction points;

[0139] Fit each right-side prediction point to obtain the right-side prediction trajectory;

[0140] The left predicted trajectory and the right predicted trajectory are forced to be parallel to obtain the target predicted trajectory of the target vehicle.

[0141] The embodiment of the present invention detects regular stationary markers on the road where the target vehicle is located through radar, and utilizes the characteristic that the curve formed by connecting the markers is parallel to the direction of the road to determine the road marker curve close to the actual trajectory of the target vehicle, and then selects a suitable curve from the road marker curve, lane line curve, curb curve and drivable area boundary curve as the first predicted trajectory, thereby realizing the fusion of multiple parameters related to the vehicle's driving trajectory, and having better stability and higher accuracy than the trajectory prediction of a single sensor.

[0142] Figure 7 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 7As shown, the electronic device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, the steps in the above-mentioned vehicle trajectory prediction method embodiments are implemented, for example Figure 2 Alternatively, when the processor 70 executes the computer program 72, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 6 Functionality of modules / units 61 to 64 shown.

[0143] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 72 in the electronic device 7. For example, the computer program 72 may be divided into Figure 6 Modules / units 61 to 64 are shown.

[0144] The electronic device 7 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will appreciate that Figure 7 It is only an example of the electronic device 7 and does not constitute a limitation of the electronic device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0145] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0146] The memory 71 may be an internal storage unit of the electronic device 7, such as a hard disk or memory of the electronic device 7. The memory 71 may also be an external storage device of the electronic device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the electronic device 7. The memory 71 is used to store the computer program and other programs and data required by the electronic device. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0147] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0148] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0149] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0150] In the embodiments provided by the present invention, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0151] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0153] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned vehicle trajectory prediction method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0154] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A vehicle trajectory prediction method, characterized in that: include: Acquire at least one first candidate trajectory curve of the target vehicle through vision; wherein the first candidate trajectory curve includes a lane line curve, a curb curve, and a drivable area boundary curve; Acquire a second candidate trajectory curve of the target vehicle by radar; wherein the second candidate curve includes a road marker curve; Selecting a road trajectory of the road where the target vehicle is located from each of the first candidate trajectory curves and the second candidate trajectory curves as a first predicted trajectory; determining a target predicted trajectory of the target vehicle based on the first predicted trajectory; The determining the target predicted trajectory of the target vehicle based on the first predicted trajectory includes: If there is no lane line curve with a confidence level higher than a preset first confidence level in each of the first candidate trajectory curves, and there is no road marker curve with a confidence level higher than a preset second confidence level in each of the second candidate trajectory curves, then obtaining a second predicted trajectory corresponding to the vehicle body posture of the target vehicle, and / or obtaining a third predicted trajectory corresponding to the steering wheel angle of the target vehicle; The first predicted trajectory, the second predicted trajectory and / or the third predicted trajectory are merged to obtain a target predicted trajectory of the target vehicle.

2. The vehicle trajectory prediction method according to claim 1, characterized in that: The selecting a road trajectory of the road where the target vehicle is located from each of the first candidate trajectory curves and the second candidate trajectory curves includes: Calculate the similarity between each first candidate trajectory curve and the second candidate trajectory curve and the previous frame prediction trajectory; Based on the confidence and similarity of each of the first candidate trajectory curves and the second candidate trajectory curves, a road trajectory of the road where the target vehicle is located is selected from each of the candidate trajectory curves.

3. The vehicle trajectory prediction method according to claim 1, characterized in that: The step of acquiring a second candidate trajectory curve of the target vehicle by using a radar comprises: Acquiring position information of multiple stationary targets detected by the radar of the target vehicle; Based on the distances between the stationary targets, the stationary targets are divided into a plurality of groups of stationary targets; Performing curve fitting on the position information of each group of stationary targets respectively to obtain a plurality of initial marker curves, and obtaining the confidence of each initial marker curve; Each initial marker curve is screened based on the confidence level to obtain a preset number of road marker curves.

4. The vehicle trajectory prediction method according to claim 3, characterized in that: The method of dividing the stationary targets into a plurality of groups of stationary targets based on the distances between the stationary targets comprises: Multiple stationary targets whose lateral distance between two adjacent stationary targets is less than a first threshold, whose longitudinal distance between two adjacent stationary targets is less than a second threshold, and whose maximum distance among all stationary targets is greater than a third threshold are divided into a group of stationary targets; wherein the first threshold is less than the second threshold, and the second threshold is less than the third threshold.

5. The vehicle trajectory prediction method according to claim 1, characterized in that: The first predicted trajectory includes a first left predicted trajectory and a first right predicted trajectory; the first predicted trajectory, the second predicted trajectory and / or the third predicted trajectory are fused to obtain a target predicted trajectory of the target vehicle, including: Move the first left predicted trajectory, and the starting point of the second predicted trajectory and / or the third predicted trajectory to the left starting point (x1, 0) of the target vehicle in the trajectory coordinate system; Taking points on the first left prediction trajectory, the second prediction trajectory and / or the third prediction trajectory according to a preset y-axis interval to obtain a plurality of left prediction points; Fit each left prediction point to obtain the left prediction trajectory; and moving the first right predicted trajectory, and the starting point of the second predicted trajectory and / or the third predicted trajectory to the right starting point (x2,0) of the target vehicle in the trajectory coordinate system; Taking points on the first right side prediction trajectory, the second prediction trajectory and / or the third prediction trajectory according to a preset y-axis interval to obtain a plurality of right side prediction points; Fit each right-side prediction point to obtain the right-side prediction trajectory; The left predicted trajectory and the right predicted trajectory are forced to be parallel to obtain a target predicted trajectory of the target vehicle.

6. A vehicle trajectory prediction device, characterized in that: include: A first acquisition module, used to acquire at least one first candidate trajectory curve of the target vehicle by vision; wherein the first candidate trajectory curve includes a lane line curve, a curb curve and a drivable area boundary curve; A second acquisition module, used to acquire a second candidate trajectory curve of the target vehicle through a radar; wherein the second candidate curve includes a road marker curve; A selection module, configured to select a road trajectory of the road where the target vehicle is located from each of the first candidate trajectory curves and the second candidate trajectory curves, and use the road trajectory as the first predicted trajectory; A prediction module, configured to determine a target predicted trajectory of the target vehicle based on the first predicted trajectory; The prediction module is specifically used for: If there is no lane line curve with a confidence level higher than a preset first confidence level in each of the first candidate trajectory curves, and there is no road marker curve with a confidence level higher than a preset second confidence level in each of the second candidate trajectory curves, then obtaining a second predicted trajectory corresponding to the vehicle body posture of the target vehicle, and / or obtaining a third predicted trajectory corresponding to the steering wheel angle of the target vehicle; The first predicted trajectory, the second predicted trajectory and / or the third predicted trajectory are merged to obtain a target predicted trajectory of the target vehicle.

7. The vehicle trajectory prediction device according to claim 6, characterized in that: The selection module is specifically used for: Calculate the similarity between each first candidate trajectory curve and the second candidate trajectory curve and the previous frame prediction trajectory; Based on the confidence and similarity of each of the first candidate trajectory curves and the second candidate trajectory curves, a road trajectory of the road where the target vehicle is located is selected from each of the candidate trajectory curves.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images