Obstacle trajectory prediction method and device, vehicle and storage medium

Abstract

The embodiment of the application provides a kind of obstacle trajectory prediction method, device, vehicle and storage medium, it is related to intelligent driving technical field.The application can be according to the history trajectory of obstacle trajectory prediction, obtain the predicted trajectory of the obstacle;Abnormal trajectory in predicted trajectory is corrected, and the corrected predicted trajectory is obtained;The trajectory smoothing processing is carried out to the corrected predicted trajectory, and the final predicted trajectory is obtained.The application can improve the accuracy, accuracy, smoothness and continuity of predicted trajectory, so as to improve the driving safety, can solve the technical problems that the current obstacle trajectory prediction method lacks the post-processing logic design of predicted trajectory, resulting in the quality of output predicted trajectory is uneven, affects the driving safety of vehicle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, and more particularly, to an obstacle trajectory prediction method and device, a vehicle, and a storage medium. BACKGROUND

[0002] Intelligent driving technology includes automatic driving technology and assisted driving technology. In intelligent driving, trajectory prediction is located at the back end of the perception module and the front end of the planning and control module, and belongs to the module that connects the two. The input of trajectory prediction is the pose information of the perceived obstacle, road structure information, and the like, trajectory prediction is used for real-time intention prediction of the perceived obstacle and real-time prediction of the future trajectory of the obstacle for a period of time, and the obtained predicted trajectory is output. The predicted trajectory can include one or more driving trajectories. The predicted trajectory of the obstacle can affect the decision-making of the intelligent driving of the vehicle, and therefore, the predicted trajectory of the obstacle is crucial to the safe driving of the vehicle.

[0003] Current obstacle trajectory prediction methods can be roughly divided into two categories. One category is to generate the future trajectory of the predicted obstacle according to the historical data representing physical actions according to a traditional physical model. The other category is a neural network prediction-based method, which generates the future trajectory according to the historical trajectory of the obstacle through encoding and decoding. The above two methods both output the future predicted trajectory for each frame of the perceived obstacle, but lack post-processing logic design for the predicted trajectory, resulting in uneven quality of the output predicted trajectory and affecting the driving safety of the vehicle. SUMMARY

[0004] Embodiments of the present application provide an obstacle trajectory prediction method, device, vehicle, and storage medium to solve the technical problem that current obstacle trajectory prediction methods lack post-processing logic design for the predicted trajectory, resulting in uneven quality of the output predicted trajectory and affecting the driving safety of the vehicle.

[0005] In a first aspect, embodiments of the present application provide an obstacle trajectory prediction method, which includes: performing trajectory prediction according to a historical trajectory of an obstacle to obtain a predicted trajectory of the obstacle; correcting an abnormal trajectory in the predicted trajectory to obtain a corrected predicted trajectory; and performing trajectory smoothing processing on the corrected predicted trajectory to obtain a final predicted trajectory.

[0006] In a second aspect, embodiments of the present application provide an obstacle trajectory prediction device, which includes: a trajectory prediction module configured to perform trajectory prediction according to a historical trajectory of an obstacle to obtain a predicted trajectory of the obstacle; a trajectory correction module configured to correct an abnormal trajectory in the predicted trajectory to obtain a corrected predicted trajectory; and a trajectory smoothing module configured to perform trajectory smoothing processing on the corrected predicted trajectory to obtain a final predicted trajectory.

[0007] In a third aspect, an embodiment of the present application provides a vehicle, comprising a memory and a processor, the memory storing an application, the application being configured to cause the processor to perform the method provided by the embodiments of the present application when invoked by the processor.

[0008] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing program codes, the program codes being configured to cause a processor to perform the method provided by the embodiments of the present application when invoked by the processor.

[0009] The obstacle trajectory prediction method provided by the embodiments of the present application has the following technical effects: after obtaining the predicted trajectory of the obstacle, the abnormal trajectory in the predicted trajectory is corrected, and the corrected predicted trajectory is smoothed. Through trajectory correction and smoothing, the accuracy, accuracy, smoothness and continuity of the predicted trajectory can be improved, thereby improving the driving safety, and solving the technical problem that the current obstacle trajectory prediction method lacks post-processing logic design of the predicted trajectory, resulting in uneven quality of the output predicted trajectory and affecting the driving safety of the vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments of the present application, all other embodiments and drawings obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0011] Figure 1 Fig. 1 shows a flowchart of an obstacle trajectory prediction method provided by an embodiment of the present application;

[0012] Figure 2 Fig. 2 shows a schematic diagram of a traffic scene provided by an exemplary embodiment of the present application;

[0013] Figure 3 Fig. 3 shows a schematic diagram of an end abnormal predicted trajectory provided by an exemplary embodiment of the present application;

[0014] Figure 4 Fig. 4 shows a schematic diagram of a trajectory obtained by correcting the end abnormal predicted trajectory provided by an exemplary embodiment of the present application; Figure 3

[0015] Figure 5 Fig. 5 shows a flowchart of an obstacle trajectory prediction method provided by another embodiment of the present application;

[0016] Figure 6 ​A structural schematic diagram of the obstacle trajectory prediction device provided by the embodiment of the application is shown.

[0017] Figure 7 A structural schematic diagram of the vehicle provided by the embodiment of the application is shown. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0019] The obstacle trajectory prediction method in the embodiments of the present application can be applied to an obstacle trajectory prediction device or a vehicle, and the obstacle trajectory prediction device can be integrated in the vehicle. The vehicle can include but is not limited to an oil car, an electric car, and a hybrid car.

[0020] Please refer to Figure 1 , Figure 1 A flowchart of the obstacle trajectory prediction method provided by an embodiment of the present application is shown. The obstacle trajectory prediction method can include steps S110 to S130.

[0021] Step S110: performing trajectory prediction according to a historical trajectory of an obstacle to obtain a predicted trajectory of the obstacle.

[0022] In the embodiments of the present application, the obstacle refers to an obstacle around the vehicle. The obstacle around the vehicle can be perceived by a perception device installed on the vehicle. The perceived obstacle is within the perception range of the perception device, and the perception device can include but is not limited to a camera, a laser radar, a millimeter wave radar, etc. Since the trajectory prediction is mainly discussed in the present application, the obstacle can refer to a dynamic obstacle, such as a pedestrian, a vehicle, etc. As an example, please refer to Figure 2 , Figure 2 A schematic diagram of a traffic scene provided by an exemplary embodiment of the present application is shown. Figure 2 In the shown traffic scene, the circular shaded area centered on the ego vehicle ① is the perception range of the ego vehicle ①, and the vehicles ②, ③, ④ and the humans ⑧, ⑨ are the obstacles that can be perceived by the current ego vehicle ①, while the vehicles ⑤, ⑥, ⑦ that are beyond the perception range of the ego vehicle ① cannot be perceived by the ego vehicle ①.

[0023] The historical trajectory of the obstacle includes the timestamp t, the pose coordinates (x i ,y i ), and the step coordinates of the historical trajectory point, etc. The historical trajectory of the obstacle can be obtained by trajectory fitting according to the kinematic information such as the speed, position, and acceleration of the historical obstacle.

[0024] The predicted trajectory of the obstacle refers to the trajectory of the obstacle in a future period of time. The predicted trajectory obtained according to the historical trajectory prediction includes at least the pose coordinates (x i ,y i ) of the trajectory points. The future period of time can be pre-set according to actual requirements, and the application does not limit the specific length of time. It can be understood that each obstacle can be predicted to have one or more predicted trajectories.

[0025] In some embodiments, the predicted trajectory of the obstacle can be obtained by performing trajectory prediction according to the historical trajectory of the obstacle in a preset period of time. The preset period of time can be pre-set according to actual requirements, for example, the preset period of time can be 2 seconds, and the application does not limit the specific preset period of time.

[0026] In some embodiments, an artificial intelligence (AI) model used for trajectory prediction can be pre-modeled and trained, and then the AI model is used to perform trajectory prediction according to the historical trajectory of the obstacle. The input of the AI model is the historical trajectory of the obstacle, and the output is the predicted trajectory of the obstacle. The AI model is modeled and trained, and after the AI model is trained, the historical trajectory of the obstacle can be input into the AI model, and the predicted trajectory of the obstacle output by the AI model can be obtained. The AI model can be modeled by using methods including but not limited to convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory (LSTM) network, trajectory clustering-based method, forward planning method, backward planning method, etc.

[0027] Step S120: correcting the abnormal trajectory in the predicted trajectory to obtain a corrected predicted trajectory.

[0028] The predicted trajectory can include an abnormal trajectory and / or a normal trajectory. The abnormal trajectory can be understood as an unreasonable trajectory, and specifically, what kind of trajectory is an abnormal trajectory can be defined in advance by the developer according to actual needs. For example, the abnormal trajectory can be defined as a predicted trajectory in which the heading angle error between adjacent trajectory points is greater than an error threshold. The abnormal trajectory can also be defined as a predicted trajectory in which the position of a trajectory point exceeds a stop line when the traffic signal light is red. The abnormal trajectory can also be defined as a predicted trajectory in which the angle difference between the trajectory direction and the direction of a low-speed obstacle is greater than an angle threshold. Abnormal trajectories defined in different ways can be screened out using different abnormal trajectory detection methods. The normal trajectory refers to other trajectories in the predicted trajectory other than the abnormal trajectory, and can also be understood as a reasonable trajectory. It can be understood that the corrected predicted trajectory in step S120 can include trajectories obtained by correcting the abnormal trajectory and the normal trajectory in the predicted trajectory.

[0029] In some embodiments, based on a plurality of abnormal trajectory detection methods, an abnormal trajectory in the predicted trajectory is obtained; and a trajectory correction method corresponding to the abnormal trajectory detection method is used to correct the obtained abnormal trajectory. The abnormal trajectory detection method and the trajectory correction method have a mapping relationship, so that different correction methods are used for different types of abnormal trajectories, thereby improving the accuracy and precision of trajectory correction. The number of abnormal trajectory detection methods can be set in advance according to actual needs, and the present application does not limit the number of abnormal trajectory detection methods. In this embodiment, the abnormal trajectory in the predicted trajectory is screened out by using a plurality of abnormal trajectory detection methods, and the abnormal trajectory is corrected by using a trajectory correction method corresponding to the abnormal trajectory detection method, so that the precision and accuracy of the predicted trajectory can be improved.

[0030] In some embodiments, the plurality of abnormal trajectory detection methods and their correction methods can include at least one of a heading angle abnormal trajectory detection method and its correction method, a terminal abnormal trajectory detection method and its correction method, a direction abnormal trajectory detection method and its correction method, and the like.

[0031] In some embodiments, the heading angle abnormal trajectory detection method can include: determining the difference between the heading angles of adjacent trajectory points in the predicted trajectory as the heading angle error between the adjacent trajectory points, i.e., delta yawi ; detecting whether the heading angle error is greater than an error threshold; if the heading angle error between the adjacent trajectory points of the predicted trajectory is greater than the error threshold, determining that the predicted trajectory is a heading angle abnormal predicted trajectory; and if the heading angle error between the adjacent trajectory points of the predicted trajectory is less than or equal to the error threshold, determining that the predicted trajectory is a normal trajectory. The error threshold can be set in advance according to the accuracy and precision requirements of trajectory prediction, for example, the error threshold can be 0.3. The heading angle abnormal trajectory correction method can include: for the heading angle abnormal predicted trajectory, using the heading angle yaw i―1Correcting a yaw angle of a next trajectory point i Specifically, for adjacent trajectory points with a yaw angle error greater than an error threshold, the yaw angle of a next trajectory point can be set as the yaw angle of a previous trajectory point, i.e., the value of the yaw angle of the previous trajectory point is assigned to the next trajectory point, so as to correct the predicted trajectory with an abnormal yaw angle and improve the accuracy and precision of the predicted trajectory.

[0032] In some embodiments, the end abnormal trajectory detection manner can include: a trajectory point at a time when the traffic signal light is red can be acquired from the predicted trajectory as a target trajectory point; it is detected whether a static obstacle (i.e., an obstacle keeping the same position) at the target trajectory point exceeds a stop line; if the static obstacle at the target trajectory point exceeds the stop line, it is determined that the predicted trajectory is an end abnormal predicted trajectory; if the static obstacle at the target trajectory point does not exceed the stop line, it is determined that the predicted trajectory is a normal trajectory. The end abnormal trajectory correction manner can include: for the end abnormal predicted trajectory, a stop line position is determined according to a relative distance between the current position of the obstacle and the stop line, and the position of the end of the predicted trajectory is set as the stop line position. The relative distance between the stop line and the current position of the obstacle can be acquired through a map interface function.

[0033] In some embodiments, a trajectory point at a time when the traffic signal light is red can be acquired from the predicted trajectory as a target trajectory point; detecting whether a static obstacle at the target trajectory point exceeds a stop line can include: the trajectory point at the time when the traffic signal light is red can be acquired from the predicted trajectory as a target trajectory point according to the time stamp of the trajectory point of the predicted trajectory; the stop line position can be acquired according to the relative distance between the current position of the obstacle and the stop line; if the static obstacle exceeds the stop line at at least one target trajectory point, it is determined that the predicted trajectory is an abnormal trajectory; if the static obstacle does not exceed the stop line at all target trajectory points, it is determined that the predicted trajectory is an end abnormal predicted trajectory.

[0034] As an example, refer to Figure 3 and Figure 4 , Figure 3 shows a schematic diagram of an end abnormal predicted trajectory provided by an example embodiment of the present application, Figure 4 shows a schematic diagram of a trajectory obtained by correcting the end abnormal predicted trajectory shown in Figure 3 . Figure 3 and Figure 4 , the vehicle in the figure is an obstacle, and the dotted line is a predicted trajectory, and the points on the dotted line are trajectory points of the predicted trajectory. It is assumed that the trajectory points at the time when the traffic signal light is red include points P1 and P2, and the vehicle will exceed the stop line at P1 and P2, so Figure 3 the predicted trajectory in the figure is an end abnormal predicted trajectory. The stop line position can be acquired, and the position of the end of the predicted trajectory is set as the stop line position. Figure 3The end position of the end abnormality predicted trajectory is modified to the stop line position, and the modified predicted trajectory is shown as Figure 4 The modified predicted trajectory is shown, and the modification of the end abnormality predicted trajectory is completed.

[0035] In some embodiments, the heading abnormality trajectory detection method can include: determining the angle difference between the heading of the predicted trajectory and the heading of the low-speed obstacle; detecting whether the angle difference between the heading of the predicted trajectory and the heading of the low-speed obstacle is greater than an angle threshold; if the angle difference between the heading of the predicted trajectory and the heading of the low-speed obstacle is greater than the angle threshold, determining that the predicted trajectory is a heading abnormality predicted trajectory; and if the angle difference between the heading of the predicted trajectory and the heading of the low-speed obstacle is less than or equal to the angle threshold, determining that the predicted trajectory is a normal trajectory. The heading abnormality trajectory modification method can include: for the heading abnormality predicted trajectory, modifying the heading of the predicted trajectory using the heading of the low-speed obstacle. The low-speed obstacle is an obstacle with a speed less than a speed threshold, and the speed threshold can be pre-set according to actual needs. For example, the speed threshold can be 5 meters per second. The angle threshold can be pre-set according to the accuracy and precision of trajectory prediction, for example, the angle threshold can be

[0036] In some embodiments, detecting whether the angle difference between the heading of the predicted trajectory and the heading of the low-speed obstacle is greater than the angle threshold to determine whether the predicted trajectory is a heading abnormality predicted trajectory can include: detecting whether the angle difference between the heading of all trajectory points in the predicted trajectory and the heading of the low-speed obstacle is greater than the angle threshold; if the angle difference between the heading of all trajectory points and the heading of the low-speed obstacle is less than or equal to the angle threshold, determining that the predicted trajectory is a normal trajectory; and if the angle difference between the heading of at least one trajectory point and the heading of the low-speed obstacle is greater than the angle threshold, determining that the predicted trajectory is a heading abnormality predicted trajectory. The heading of the low-speed obstacle can include the heading of the obstacle or the speed heading of the obstacle. For example, if the obstacle is a vehicle, the heading of the low-speed vehicle can include the heading of the vehicle head or the speed heading of the vehicle.

[0037] After the abnormal trajectory is modified, the trajectories of all predicted trajectories are relatively smooth and the differences between them are small. In order to further improve the smoothness, continuity and accuracy of the predicted trajectory, step S130 of trajectory smoothing can be performed.

[0038] Step S130: performing trajectory smoothing processing on the modified predicted trajectory to obtain the final predicted trajectory.

[0039] A polynomial fitting method can be used to perform trajectory smoothing processing on the modified predicted trajectory. In some embodiments, based on the trajectory points of the modified predicted trajectory, a quintic polynomial can be used to perform trajectory smoothing processing on the modified predicted trajectory to obtain the final predicted trajectory.

[0040] For example, the quintic polynomial can be expressed as follows:

[0041] x = a 11 *t 5 +a 12 *t 4 +a 13 *t 3 +a 14 *t 4 +a 15 *t 5

[0042] y = a 21 *t 5 +a 22 *t 4 +a 23 *t 3 +a 24 *t 4 +a 25 *t 5

[0043] wherein t represents a timestamp of a trajectory point of the corrected prediction trajectory; x represents an abscissa of the trajectory point; y represents an ordinate of the trajectory point; a 11 , a 12 , a 13 , a 14 , a 15 , a 21 , a 22 , a 23 , a 24 , a 25 are to-be-solved coefficients, which can be solved by bringing the trajectory points of the corrected prediction trajectory into the above-mentioned quintic polynomial, so as to obtain an expression of the final prediction trajectory.

[0044] In the embodiments of the present application, the quintic polynomial can be used for trajectory smoothing fitting, the fitting effect is good, and the accuracy, smoothness and continuity of the prediction trajectory of the obstacle can be further improved, a high-quality prediction trajectory is provided for the downstream, and driving safety is ensured.

[0045] In some embodiments, the final prediction trajectory of the obstacle can be output to a downstream module, so as to realize related downstream functions based on the final prediction trajectory of the obstacle, and improve the performance of intelligent driving functions. The downstream module can include but is not limited to a collision detection module, a local map updating module, or a route planning and control module, etc.

[0046] Steps S110 to S130 have the following technical effects: after obtaining the predicted trajectory of the obstacle, the abnormal trajectory in the predicted trajectory is corrected, and the corrected predicted trajectory is trajectory smoothed. Through trajectory correction and smoothing, the accuracy, accuracy, smoothness and continuity of the predicted trajectory can be improved, thereby improving the driving safety, and solving the technical problem that the current obstacle trajectory prediction method lacks post-processing logic design of the predicted trajectory, resulting in uneven quality of the output predicted trajectory, affecting the driving safety of the vehicle.

[0047] Referring to Figure 5 , Figure 5 A flowchart of an obstacle trajectory prediction method provided by another embodiment of the present application is shown. The obstacle trajectory prediction method can include steps S210 to S280.

[0048] Step S210: trajectory prediction is performed according to the historical trajectory of the obstacle, and a predicted trajectory of the obstacle is obtained. For specific description of step S210, please refer to step S110.

[0049] Step S220: according to the pose of the trajectory point of the predicted trajectory, the speed, heading angle, acceleration and angular velocity of the trajectory point are determined.

[0050] As shown in the following expression, the speed v i and the heading angle yaw i of the trajectory point can be calculated respectively according to the pose coordinates (x i ,y i ) of the trajectory point of the predicted trajectory; then the acceleration a i of the trajectory point is calculated according to the speed of the trajectory point, and the angular velocity w i of the trajectory point is calculated according to the heading angle of the trajectory point.

[0051]

[0052] Step S230: according to the speed and acceleration of the trajectory point, the longitudinal intention of the obstacle is determined.

[0053] In some embodiments, the longitudinal intention of the obstacle can be first determined according to the speed of the trajectory point, and when the longitudinal intention of the obstacle cannot be determined according to the speed of the trajectory point, the longitudinal intention of the obstacle is determined according to the acceleration of the trajectory point.

[0054] In some embodiments, the longitudinal intention of the obstacle can be determined according to the speed of the trajectory point, and the longitudinal intention of the obstacle can be determined according to the acceleration of the trajectory point, and then the longitudinal intention of the obstacle is comprehensively analyzed by combining the two determination results.

[0055] In some embodiments, the longitudinal intention of the obstacle is determined according to the speed of the trajectory points. Specifically, the longitudinal intention of the obstacle can be determined according to the speed v0 of the first trajectory point (i.e. the current speed of the obstacle) and the speed v of the last trajectory point of the predicted trajectory. last Specifically, the first speed, the second speed, the first multiple, and the second multiple can be pre-set according to actual needs, wherein the first speed is greater than the second speed, and the first multiple is less than the second multiple. The product of the first multiple and the speed of the first trajectory point can be calculated as a third speed, and the product of the second multiple and the speed of the last trajectory point can be calculated as a fourth speed. Then the speed of the first trajectory point is compared with the first speed, the second speed, and the fourth speed respectively to obtain a speed comparison result of the first trajectory point, and the speed of the last trajectory point is compared with the first speed, the second speed, and the third speed respectively to obtain a speed comparison result of the second trajectory point. The longitudinal intention of the predicted trajectory of the obstacle is determined according to the speed comparison results of the first trajectory point and the last trajectory point.

[0056] For example, as shown in Table 1, assuming that the first speed is 0.6 km / h (kilometers per hour), the second speed is 0.3 km / h, the first multiple is 0.8, and the second multiple is 1.2, the longitudinal intention of the predicted trajectory of the obstacle can be determined according to the speed comparison results of the first trajectory point and the last trajectory point as shown in Table 1. If the speed of the first trajectory point and the last trajectory point is less than the first speed, it can be determined that the longitudinal intention of the predicted trajectory of the obstacle is stationary. If the speed of the first trajectory point is greater than or equal to the first speed and the speed of the last trajectory point is less than the first speed, it can be determined that the longitudinal intention of the predicted trajectory of the obstacle is deceleration. If the speed of the first trajectory point is greater than the second speed and the speed of the last trajectory point is greater than or equal to the first speed and the speed of the last trajectory point is less than or equal to the first multiple of the speed of the first trajectory point, it can be determined that the longitudinal intention of the predicted trajectory of the obstacle is deceleration. If the speed of the first trajectory point is less than the first speed and the speed of the last trajectory point is greater than the first speed, it can be determined that the longitudinal intention of the predicted trajectory of the obstacle is start. If the speed of the first trajectory point is greater than or equal to the first speed and the speed of the first trajectory point is less than or equal to the second speed and the speed of the last trajectory point is greater than the second multiple of the speed of the first trajectory point and the speed of the last trajectory point is greater than or equal to the second speed, it can be determined that the longitudinal intention of the predicted trajectory of the obstacle is start. If the speed of the first trajectory point is greater than or equal to the second speed and the speed of the last trajectory point is greater than or equal to the second multiple of the speed of the first trajectory point, it can be determined that the longitudinal intention of the predicted trajectory of the obstacle is acceleration. If none of the above conditions is met, it can be determined that the longitudinal intention of the obstacle cannot be determined according to the speed of the trajectory points.

[0057] Table 1

[0058]

[0059] In some embodiments, the longitudinal intention of the obstacle is determined according to the acceleration of the trajectory point. Specifically, the longitudinal intention of the obstacle can be determined according to the minimum acceleration a min and the maximum acceleration a max of the trajectory point. Specifically, the first acceleration and the second acceleration can be pre-set according to actual needs, and the values of the first acceleration and the second acceleration are the same but the signs are just opposite. The maximum acceleration can be compared with the first acceleration and 0 respectively to obtain the comparison results of the maximum acceleration. The minimum acceleration can be compared with 0 and the second acceleration respectively to obtain the comparison results of the minimum acceleration. The longitudinal intention of the predicted trajectory of the obstacle is determined according to the comparison results of the maximum acceleration and the minimum acceleration.

[0060] For example, referring to Table 2, it is assumed that the first acceleration is 0.3 meters per second squared (m / s 2 ) and the second acceleration is -0.3 m / s 2 . The longitudinal intention of the predicted trajectory of the obstacle is determined according to the comparison results of the maximum acceleration and the minimum acceleration, which can be shown in Table 2. If the maximum acceleration is greater than the first acceleration and the minimum acceleration is greater than or equal to 0, it is determined that the longitudinal intention of the predicted trajectory of the obstacle is acceleration; if the maximum acceleration is less than or equal to 0 and the minimum acceleration is less than the first acceleration, it is determined that the longitudinal intention of the predicted trajectory of the obstacle is deceleration; if neither of the above two conditions is met, it is determined that the longitudinal intention of the predicted trajectory of the obstacle is constant speed.

[0061] Table 2

[0062] Acceleration comparison result Longitudinal intention [a max >0.3 and a min ≥0]]> Accelerating a max ≤0 and a min <―0.3]] Decelerating None of the above conditions are met Constant speed

[0063] Step S240: determining the lateral intention of the obstacle according to the heading angle and the angular velocity of the trajectory point.

[0064] In some embodiments, the lateral intention of the obstacle can be determined according to the heading angle of the trajectory point.

[0065] In some embodiments, the lateral intention of the obstacle can be determined according to the heading angle of the trajectory point and the angular velocity of the trajectory point, and the lateral intention of the obstacle is determined by combining the two determination results.

[0066] For example, referring to the expression below this paragraph, the lateral intention of the obstacle is determined according to the heading angle of the trajectory point, and specifically, the heading angle yaw lastdelt yaw yaw0

[0067] delt yaw yaw0 last yaw0

[0068] Specifically, the first angle, the second angle, the third angle and the fourth angle can be set according to actual requirements in advance, the heading angle change amount is compared with the first angle, the second angle, the third angle and the fourth angle respectively to obtain a comparison result of the heading angle change amount, and the lateral intention of the predicted trajectory of the obstacle is determined according to the comparison result of the heading angle change amount.

[0069] For example, referring to Table 3, it is assumed that the first angle is the second angle the third angle π, and the fourth angle According to the comparison result of the heading angle change amount, the lateral intention of the predicted trajectory of the obstacle can be determined as shown in Table 3. If the heading angle change amount is greater than the first angle and less than the second angle, it can be determined that the lateral intention of the predicted trajectory of the obstacle is left turn; if the heading angle change amount is greater than the third angle and less than the fourth angle, it can be determined that the lateral intention of the predicted trajectory of the obstacle is right turn; if the heading angle change amount is less than the third angle or greater than the second angle, it can be determined that the lateral intention of the predicted trajectory of the obstacle is U-turn; if none of the above conditions is met, it can be determined that the lateral intention of the predicted trajectory of the obstacle is lane keeping.

[0070] Table 3

[0071]

[0072] The lateral intention of the obstacle can be determined according to the angular velocities of the trajectory points. Specifically, the lateral intention of the obstacle can be determined according to the angular velocity directions and the angular velocity difference of the first trajectory point and the last trajectory point. For example, if the angular velocity directions of the first trajectory point and the last trajectory point are both clockwise and the angular velocity difference is less than or equal to an angular velocity threshold, it can be determined that the lateral intention of the predicted trajectory of the obstacle is to turn right; if the angular velocity directions of the first trajectory point and the last trajectory point are both clockwise and the angular velocity difference is greater than the angular velocity threshold, it can be determined that the lateral intention of the predicted trajectory of the obstacle is to turn right at a crossroad; if the angular velocity directions of the first trajectory point and the last trajectory point are both counterclockwise and the angular velocity difference is less than or equal to the angular velocity threshold, it can be determined that the lateral intention of the predicted trajectory of the obstacle is to turn left; if the angular velocity directions of the first trajectory point and the last trajectory point are both counterclockwise and the angular velocity difference is greater than the angular velocity threshold, it can be determined that the lateral intention of the predicted trajectory of the obstacle is to turn left at a crossroad; and if none of the above conditions is met, it can be determined that the lateral intention of the predicted trajectory of the obstacle is to keep the lane.

[0073] Step S250: The predicted trajectory with the abnormal heading angle is corrected.

[0074] Step S260: The predicted trajectory with the abnormal heading angle is corrected.

[0075] Step S270: The predicted trajectory with the abnormal end is corrected.

[0076] Step S280: The modified predicted trajectory is subjected to trajectory smoothing processing.

[0077] For specific descriptions of steps S250 to S280, please refer to steps S120 to S130 described above.

[0078] Steps S210 to S280 have the following technical effects: kinematic information such as the speed, acceleration, and heading angle of the trajectory point can be obtained by inversely analyzing the predicted trajectory, the lateral intention and the longitudinal intention of the predicted trajectory can be obtained by analyzing and solving the kinematic information, the accuracy of the intention prediction of the obstacle can be improved, the vehicle can better understand the surrounding environment and make safer decisions, the safety and intelligence of intelligent driving can be improved, and human-machine collaborative driving can be promoted. In addition, based on the kinematic characteristics of the estimated point of the predicted trajectory, abnormal / irrational predicted trajectories can be screened out and corrected, and the corrected predicted trajectory can be smoothed, the accuracy, accuracy, smoothness, continuity, and fitting effect of the predicted trajectory can be improved through trajectory correction and smoothing, the quality of the predicted trajectory of the obstacle received by the downstream can be ensured, the driving safety can be improved, and the technical problem that the current obstacle trajectory prediction method lacks post-processing logic design of the predicted trajectory, resulting in uneven quality of the output predicted trajectory and affecting the driving safety of the vehicle can be solved.

[0079] Please refer to Figure 6 , Figure 6 A structure diagram of an obstacle trajectory prediction device provided by an embodiment of the present application is shown. The obstacle trajectory prediction device 100 can be integrated in a vehicle. The obstacle trajectory prediction device 100 can include a trajectory prediction module 110, a trajectory correction module 120, and a trajectory smoothing module 130.

[0080] The trajectory prediction module 110 is configured to perform trajectory prediction based on the historical trajectory of the obstacle to obtain a predicted trajectory of the obstacle.

[0081] The trajectory correction module 120 is configured to correct an abnormal trajectory in the predicted trajectory to obtain a corrected predicted trajectory.

[0082] The trajectory smoothing module 130 is configured to perform trajectory smoothing processing on the corrected predicted trajectory to obtain a final predicted trajectory.

[0083] In some embodiments, the trajectory correction module 120 is further configured to obtain an abnormal trajectory in the predicted trajectory based on a plurality of abnormal trajectory detection methods, and correct the obtained abnormal trajectory by using a trajectory correction method corresponding to the abnormal trajectory detection method.

[0084] In some embodiments, the trajectory correction module 120 is further configured to determine that the predicted trajectory is a heading angle abnormal predicted trajectory if the heading angle error between adjacent trajectory points of the predicted trajectory is greater than an error threshold, and correct the heading angle of the next trajectory point by using the heading angle of the previous trajectory point for the heading angle abnormal predicted trajectory.

[0085] In some embodiments, the trajectory correction module 120 is further configured to determine that the predicted trajectory is an end abnormal predicted trajectory if a position of a trajectory point of the predicted trajectory exceeds a stop line when the traffic signal is red; and determine the position of the stop line according to a relative distance between a current position of the obstacle and the stop line, and set the position of the end of the predicted trajectory as the position of the stop line.

[0086] In some embodiments, the trajectory correction module 120 is further configured to determine that the predicted trajectory is an orientation abnormal predicted trajectory if an angle difference between an orientation of the predicted trajectory and an orientation of a low-speed obstacle is greater than an angle threshold, where the low-speed obstacle is an obstacle with a speed less than a speed threshold; and correct the orientation of the predicted trajectory using the orientation of the low-speed obstacle for the orientation abnormal predicted trajectory.

[0087] In some embodiments, the trajectory smoothing module 130 is further configured to perform trajectory smoothing processing on the corrected predicted trajectory using a quintic polynomial to obtain a final predicted trajectory.

[0088] In some embodiments, the trajectory prediction module 110 is further configured to determine a speed, a heading angle, an acceleration, and an angular velocity of a trajectory point according to a pose of the trajectory point of the predicted trajectory; determine a longitudinal intention of the obstacle according to the speed and the acceleration of the trajectory point; and determine a lateral intention of the obstacle according to the heading angle and the angular velocity of the trajectory point.

[0089] It can be clearly understood by those skilled in the art that the above device provided by the embodiments of the present application can realize the method provided by the embodiments of the present application. The specific working process of the above-described device and module can refer to the process corresponding to the method in the embodiments of the present application, which will not be described here.

[0090] In the embodiments provided in the present application, the coupling, direct coupling or communication connection between the modules displayed or discussed can be indirect coupling or communication coupling through some interfaces, devices or modules, and can be electrical, mechanical or other forms, which are not specifically limited in the embodiments of the present application.

[0091] In addition, each functional module in the embodiments of the present application can be integrated in one processing module, or each module can exist physically independently, or two or more modules can be integrated in one module. The above integrated module can be realized in the form of hardware or in the form of a software functional module.

[0092] Please refer to Figure 7 , Figure 7A structural schematic diagram of a vehicle is shown. The vehicle 200 can include a memory 210 and a processor 220, the memory 210 storing an application configured to cause the processor to perform the method provided by the embodiments of the application when invoked by the processor 220.

[0093] The processor 220 can include one or more processing cores. The processor 220 connects various parts within the vehicle 200 with various interfaces and lines, for running or executing instructions, programs, code sets or instruction sets stored in the memory 210, and invoking running or executing data stored in the memory 210, to perform various functions and process data of the vehicle 200.

[0094] The processor 220 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 220 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU is mainly used to process the operating system, user interface and application programs, etc.; the GPU is used to render and draw display content; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated in the processor 220, but can be realized by a separate communication chip.

[0095] The memory 210 can include a random access memory (RAM) and can also include a read-only memory (ROM). The memory 210 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 210 can include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the above-mentioned various method embodiments, etc. The data storage area can store data created by the vehicle 200 in use, etc.

[0096] The embodiments of the present application further provide a computer readable storage medium, which stores program codes configured to execute the method provided by the embodiments of the present application when called by a processor. The computer readable storage medium can be an electronic storage medium such as a flash memory, an Electrically-Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a hard disk or a ROM.

[0097] In some embodiments, the computer readable storage medium comprises a Non-Transitory Computer-Readable Storage Medium (Non-TCRSM). The computer readable storage medium has storage space for program codes to execute any of the method steps described above. The program codes can be read from or written to one or more computer program products. The program codes can be compressed in a suitable form.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of obstacle trajectory prediction, the method comprising: receiving a plurality of images of a scene; determining a plurality of obstacle trajectories based on the plurality of images; and outputting the plurality of obstacle trajectories. The method comprises the following steps: trajectory prediction is performed according to a historical trajectory of an obstacle, and a predicted trajectory of the obstacle is obtained; an abnormal trajectory in the predicted trajectory is obtained based on a plurality of abnormal trajectory detection methods, wherein the abnormal trajectory in the predicted trajectory is obtained based on the plurality of abnormal trajectory detection methods, which comprises: determining a heading angle error between adjacent trajectory points in the predicted trajectory as a difference value of the heading angles between the adjacent trajectory points; if the heading angle error between the adjacent trajectory points in the predicted trajectory is greater than an error threshold, it is determined that the predicted trajectory is a predicted trajectory with abnormal heading angle; a trajectory correction method corresponding to the abnormal trajectory detection method is used to correct the obtained abnormal trajectory, wherein the trajectory correction method corresponding to the abnormal trajectory detection method is used to correct the obtained abnormal trajectory, which comprises: for the predicted trajectory with abnormal heading angle, the heading angle of the next trajectory point is corrected using the heading angle of the previous trajectory point; the corrected predicted trajectory is subjected to trajectory smoothing processing to obtain a final predicted trajectory.

2. The method of claim 1, wherein, The abnormal trajectory in the predicted trajectory is obtained based on the plurality of abnormal trajectory detection methods, which further comprises: a trajectory point when a traffic signal light is red in the predicted trajectory is obtained as a target trajectory point; if a stationary obstacle at the target trajectory point exceeds a stop line, it is determined that the predicted trajectory is a predicted trajectory with an end abnormality; the trajectory correction method corresponding to the abnormal trajectory detection method is used to correct the obtained abnormal trajectory, which further comprises: for the predicted trajectory with an end abnormality, the position of the stop line is determined according to the relative distance between the current position of the obstacle and the stop line, and the position of the end of the predicted trajectory is set as the position of the stop line.

3. The method of claim 1, wherein, The abnormal trajectory in the predicted trajectory is obtained based on the plurality of abnormal trajectory detection methods, which further comprises: an angle difference between the direction of the predicted trajectory and the direction of a low-speed obstacle is determined, the low-speed obstacle being an obstacle with a speed less than a speed threshold; if the angle difference between the direction of the predicted trajectory and the direction of the low-speed obstacle is greater than an angle threshold, it is determined that the predicted trajectory is a predicted trajectory with abnormal direction; the trajectory correction method corresponding to the abnormal trajectory detection method is used to correct the obtained abnormal trajectory, which further comprises: for the predicted trajectory with abnormal direction, the direction of the predicted trajectory is corrected using the direction of the low-speed obstacle.

4. The method of claim 1, wherein, The trajectory smoothing processing is performed on the corrected predicted trajectory to obtain the final predicted trajectory, which comprises: a quintic polynomial is used to perform trajectory smoothing processing on the corrected predicted trajectory to obtain the final predicted trajectory.

5. The method of claim 1, wherein, After the trajectory prediction is performed according to the historical trajectory of the obstacle to obtain the predicted trajectory of the obstacle, the method further comprises: the speed, the heading angle, the acceleration and the angular velocity of a trajectory point are determined according to the pose of the trajectory point; the longitudinal intention of the obstacle is determined according to the speed and the acceleration of the trajectory point; the lateral intention of the obstacle is determined according to the heading angle and the angular velocity of the trajectory point.

6. An obstacle trajectory prediction device characterized by comprising: The method comprises the following steps: a trajectory prediction module is configured to perform trajectory prediction according to a historical trajectory of an obstacle, and obtain a predicted trajectory of the obstacle; The trajectory correction module is configured to: acquire an abnormal trajectory in the predicted trajectory based on a plurality of abnormal trajectory detection manners; and correct the acquired abnormal trajectory by using a trajectory correction manner corresponding to the abnormal trajectory detection manner. The trajectory smoothing module is configured to perform trajectory smoothing processing on the corrected predicted trajectory to obtain a final predicted trajectory.

7. A vehicle characterized by comprising: The memory and the processor are configured to store an application program on the memory, and the application program is configured to enable the processor to perform the method of any one of claims 1-5 when the application program is called by the processor. The computer-readable storage medium stores a program code, and the program code is configured to enable the processor to perform the method of any one of claims 1-5 when the program code is called by the processor.

8. A computer readable storage medium, characterized in that, ​

Images