Track planning method and device for automatic driving, electronic equipment and medium

Abstract

The invention provides a trajectory planning method and device for automatic driving, electronic equipment and a medium. According to one example of the invention, the method can be applied to an end-to-end automatic driving model, and the method can comprise the steps: extracting lane line features from aerial view features corresponding to a current road image, and constructing lane line information in a vector map; wherein the current road image is an image of a road where the self-driving vehicle runs; under the condition that the planned self-vehicle track intersects with the lane line, path points in the self-vehicle track are adjusted based on intersection points; the self-vehicle track is the driving track of the self-vehicle in the future preset time.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an autonomous driving trajectory planning method, device, electronic device and medium. Background Art

[0002] With the rapid development of deep learning technology, various tasks in the field of intelligent driving, such as target detection and tracking, based on deep learning algorithms have achieved excellent results. Most industrial solutions deploy each task as a separate model. Although this reduces the difficulty of cross-team research and development, since the optimization goals of each task are independent, there are also risks such as cross-module information loss and loss accumulation. Therefore, the academic community has gradually shifted its research focus to end-to-end autonomous driving models.

[0003] The end-to-end autonomous driving model can achieve direct mapping from sensor input to vehicle control output through a single neural network model, imitating the intuitive way human drivers perceive the environment and respond.

[0004] The current end-to-end autonomous driving model can only predict the driving trajectory in accordance with the perception results, that is, the planned path will not cause collisions. However, the driving trajectory may not comply with traffic regulations and there are certain safety risks. Summary of the invention

[0005] In order to overcome the problems existing in the related technologies, the present application provides a trajectory planning method, device, electronic device and medium for autonomous driving.

[0006] According to a first aspect of any embodiment of the present application, a trajectory planning method for autonomous driving is provided, the method being applied to an end-to-end autonomous driving model, the method comprising:

[0007] Lane line features are extracted from the bird's-eye view features corresponding to the current road image, and lane line information in the vector map is constructed; wherein the current road image is an image of the road on which the autonomous driving vehicle is traveling; when the planned vehicle trajectory intersects with the lane line, the path points in the vehicle trajectory are adjusted based on the intersection; the vehicle trajectory is the driving trajectory of the vehicle within a preset time in the future.

[0008] According to a second aspect of any embodiment of the present application, there is provided a trajectory planning device for autonomous driving, the device being applied to an end-to-end autonomous driving model, the device comprising:

[0009] A lane line construction module is used to extract lane line features from the bird's-eye view features corresponding to the current road image, and construct lane line information in the vector map; wherein the current road image is an image of the road on which the autonomous driving vehicle is traveling;

[0010] The vehicle trajectory adjustment module is used to adjust the path points in the vehicle trajectory based on the intersection point when the planned vehicle trajectory intersects with the lane line; the vehicle trajectory is the driving trajectory of the vehicle within a preset time in the future.

[0011] According to a third aspect of any embodiment of the present application, there is provided an electronic device, including:

[0012] processor;

[0013] a memory for storing processor-executable instructions;

[0014] The processor implements the method described in any embodiment of the present application by running the executable instructions.

[0015] According to a fourth aspect of any embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the method described in any embodiment of the present application is implemented.

[0016] According to a fifth aspect of any embodiment of the present application, a computer program product is provided, on which a computer program / instruction is stored, and when the computer program / instruction is executed by a processor, the method described in any embodiment of the present application is implemented.

[0017] The technical solution provided by this application may have the following beneficial effects:

[0018] According to the above embodiments, it can be seen that by extracting lane line features from the bird's-eye view features corresponding to the current road image on which the autonomous driving vehicle is traveling, the lane line information in the vector map is constructed online, and the lane lines in the lane line information are used to timely adjust the path points in the vehicle trajectory based on the intersection when the planned vehicle trajectory intersects with the lane lines that are prohibited from intersecting with the vehicle in the road traffic rules. This can ensure that the vehicle trajectory within a preset time in the future complies with the traffic rules, thereby improving the compliance and safety of autonomous driving.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in the specification and constitute a part of this application, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0021] Figure 1 is a flow chart of a trajectory planning method for autonomous driving according to an exemplary embodiment of the present application;

[0022] Figure 2is a schematic diagram of adjusting the trajectory of a vehicle by translation according to an exemplary embodiment of the present application;

[0023] Figure 3 is a schematic diagram of a method of decelerating and adjusting a vehicle trajectory according to an exemplary embodiment of the present application;

[0024] Figure 4 is a flow chart of another automatic driving trajectory planning method according to an exemplary embodiment of the present application;

[0025] Figure 5 is a structural schematic diagram of an electronic device according to an exemplary embodiment of the present application;

[0026] Figure 6 It is a block diagram of a trajectory planning device for autonomous driving according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0028] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0030] The driving trajectory predicted by the current end-to-end autonomous driving model can only conform to the perception results, and cannot guarantee that the driving trajectory complies with traffic regulations, which poses certain safety risks.

[0031] In order to solve the above problems, the present application proposes a trajectory planning method for autonomous driving. To further illustrate the present application, the following embodiments are provided:

[0032] See also Figure 1 , Figure 1 The present application is a flowchart of a trajectory planning method for autonomous driving according to an exemplary embodiment. The trajectory planning method for autonomous driving is applied to an end-to-end autonomous driving model.

[0033] The end-to-end autonomous driving model is a neural network model that uses deep learning technology to learn driving behavior from raw data such as images, radar signals, and lidar signals. It can directly convert sensor inputs into vehicle steering, acceleration, braking and other control instructions.

[0034] The automatic driving trajectory planning method may include the following steps:

[0035] Step 102: extracting lane line features from the bird's-eye view features corresponding to the current road image, and constructing lane line information in the vector map; wherein the current road image is an image of the road on which the autonomous driving vehicle is traveling.

[0036] In this step, the end-to-end autonomous driving model can collect the current road image of the road on which the autonomous driving vehicle is traveling in real time through sensors such as on-board cameras.

[0037] Using deep learning algorithms such as Convolutional Neural Networks (CNN), features of the collected current road image are extracted to obtain a feature map containing key information such as lane lines.

[0038] The feature map is converted into a bird's-eye view feature through perspective transformation (Perspective Transformation) or inverse perspective transformation (Inverse Perspective Mapping, IPM) to better represent the road layout and lane line information.

[0039] In the bird's-eye view features, lane line features are extracted using surround lane line map extraction methods, straight line detection based on Hough transform and other lane line detection algorithms. The extracted lane line features are converted into vector format so that they can be represented in the vector map, and a vector map with secondary classification of lane line information such as lane line categories and road layout is constructed online.

[0040] The current road image is an image of the road on which the autonomous driving vehicle is traveling. The lane line features are various attribute information of the lane lines on the road surface, such as line color, line type, etc. The line color can be white, yellow, etc., and the line type can be solid line, dotted line, etc.

[0041] Lane line information is used to identify lane lines in the vector map. Lane line information may include: lane line category. Lane line information may also include: lane line location information, lane line shape and other information.

[0042] Lane line categories are used to determine lane lines that are prohibited from intersecting with vehicles in road traffic regulations. Lane line categories can be white double solid lines, yellow dashed lines, etc. Lane lines are lines that are prohibited from intersecting with vehicles in road traffic regulations, such as white double solid lines, yellow double solid lines, etc.

[0043] Step 104: When the planned ego vehicle trajectory intersects with the lane line, adjust the path points in the ego vehicle trajectory based on the intersection point; the ego vehicle trajectory is the driving trajectory of the ego vehicle within a preset time in the future.

[0044] In this step, the end-to-end autonomous driving model can determine the lane lines in the vector map that are prohibited from intersecting with the vehicle based on the lane line categories in the vector map, and optimize the planned self-vehicle trajectory.

[0045] The end-to-end autonomous driving model can determine the intersection of the planned ego vehicle trajectory with the lane line. When the ego vehicle trajectory intersects with the lane line, the path points in the ego vehicle trajectory are adjusted based on the intersection point between the ego vehicle trajectory and the lane line through local optimization.

[0046] For example, the path points in the vehicle trajectory that intersect with the lane line and the path points after the intersection are moved or deleted to avoid the vehicle trajectory from intersecting with the lane line.

[0047] The self-driving vehicle trajectory is the driving trajectory of the self-driving vehicle within a preset time in the future. The waypoint refers to the waypoint set on the self-driving vehicle trajectory to describe and define the trajectory, which is used to guide the self-driving vehicle to drive according to the planned self-driving vehicle trajectory. The waypoint may include the position information of the self-driving vehicle in space.

[0048] In one embodiment, before determining whether the planned ego vehicle trajectory intersects with the training lane line, the end-to-end autonomous driving model may be trained. The training module in the end-to-end autonomous driving model may obtain the initial ego vehicle trajectory generated by the planning module and determine whether the initial ego vehicle trajectory intersects with the training lane line.

[0049] If the initial ego vehicle trajectory intersects with the training lane line, obtain the number of points and the first angle of the intersection path points in the initial ego vehicle trajectory. Calculate the loss value of the loss function based on the number of points and the first angle.

[0050] Among them, the larger the value of the number of point sets and the first angle, the more intersection path points there are in the initial ego-vehicle trajectory, the larger the intersection angle between the initial ego-vehicle trajectory and the training lane line, and the larger the loss value.

[0051] Based on the calculated loss value, the parameters of the planning module are updated using gradient descent or other optimization algorithms. The above training process is repeated to minimize the loss value and obtain the trained end-to-end autonomous driving model. The trained end-to-end autonomous driving model can obtain the prior knowledge that "the output trajectory of the vehicle cannot intersect with the training lane line".

[0052] The intersection path point is the path point in the initial ego vehicle trajectory that intersects with the training lane line. The number of point sets is the total number of intersection path points in the initial ego vehicle trajectory. The training lane line is a preset road element that is prohibited from intersecting with the vehicle in the road traffic rules.

[0053] The first angle is the angle formed by the tangent direction of the initial vehicle trajectory at the intersection path point and the tangent direction of the training lane line, and the range of the first angle can be 0° to 180°.

[0054] The larger the first angle is within the range of 0° to 180°, the higher the degree of deviation between the vehicle's driving direction and the driving lane is; the smaller the first angle is within the range of 0° to 180°, the lower the degree of deviation between the vehicle's driving direction and the driving lane is.

[0055] The main task of the planning module is to plan a suitable trajectory for the ego vehicle based on the navigation information and the current state of the vehicle. The initial trajectory is the trajectory of the ego vehicle output by the planning module before training.

[0056] The loss function may be a piecewise loss function, which means that the greater the value of the number of point sets and the value of the first angle in the range of 0° to 180°, the greater the loss value. Exemplarily, the loss value may be calculated according to the following formula (1):

[0057] a(sin(angle)-cos(angle))L2+(1-a)n(1)

[0058] Where a represents the preset proportional coefficient, which is used to set the proportion of L2 loss and the proportion of the number of intersections in the loss function. angle represents the first angle, and n represents the number of point sets.

[0059] L2 represents the L2 loss of the vector map point set and the ego vehicle trajectory point set that forms the ego vehicle trajectory. The vector map point set is the point located on the training lane line that intersects the ego vehicle trajectory and the distance between the ego vehicle and the ego vehicle is within a preset range, which can be 20 meters, 25 meters, etc.

[0060] As described above, if the initial ego-vehicle trajectory generated by the planning module intersects with the training lane line, the loss value of the loss function is calculated according to the number of point sets of the intersecting path points and the first angle, and based on the loss value, the parameters of the planning module are updated to obtain the trained planning module, and the trained end-to-end autonomous driving model is obtained to ensure that the ego-vehicle trajectory output by the planning module in the trained end-to-end autonomous driving model can minimize the path points that intersect with the training lane lines, and more accurately generate the ego-vehicle trajectory that meets the lane line regulations.

[0061] The autonomous driving trajectory planning method in this embodiment extracts lane line features from the bird's-eye view features corresponding to the current road image on which the autonomous driving ego vehicle is traveling, constructs lane line information in the vector map online, and uses the lane lines in the lane line information to timely adjust the path points in the ego vehicle trajectory based on the intersection points when the planned ego vehicle trajectory intersects with lane lines that are prohibited from intersecting with vehicles in road traffic rules. This can ensure that the ego vehicle trajectory within a preset time in the future complies with traffic regulations, thereby improving the compliance and safety of autonomous driving.

[0062] In addition, the end-to-end autonomous driving model can integrate tasks such as extracting lane line features, building vector maps, planning and adjusting the vehicle trajectory into a whole, and optimize the entire model with planning as the guide, avoiding risks such as information loss and loss accumulation caused by independent task execution.

[0063] In the above-mentioned embodiment, it is introduced that the end-to-end autonomous driving model extracts lane line features from the bird's-eye view features corresponding to the current road image, constructs lane line information in the vector map, and adjusts the path points in the self-vehicle trajectory when the planned self-vehicle trajectory intersects with the lane line. In the following embodiment, the process of constructing the lane line information in the vector map will be described in more detail and can be applied to any of the above embodiments.

[0064] In one embodiment, the end-to-end autonomous driving model may include a vector map perception module and a lane line construction model. The modular design in the end-to-end autonomous driving model can decouple the modules.

[0065] Other modules in the end-to-end autonomous driving model can first obtain bird's-eye view features from the current road map. The vector map perception module can obtain the bird's-eye view features and input the bird's-eye view features into the lane line construction model.

[0066] The lane line construction model can model the map elements as a point set with a set of equivalent permutations through a unified permutation equivalence modeling method, which can accurately describe the shape of the map elements and stabilize the learning process.

[0067] The lane construction model also features a hierarchical query embedding scheme to flexibly encode instance-level and point-level information, which can perform hierarchical bipartite matching for map element learning and supervise point- and edge-level geometry using the proposed point-to-point loss and edge direction loss.

[0068] The lane construction model can extract the vector lane features from the bird's-eye view features based on the permutation equivalence modeling method and the hierarchical query embedding scheme and convert the lane features into a vector map with secondary categories, and output the lane categories through the category classification head in the lane construction model. The vector map perception module can obtain the vector map including the lane categories output by the lane construction model.

[0069] The lane line construction model is an online vector map construction model that constructs lane line information such as lane line type, which can be a Transformer-based model. The category classification head is used to output the lane line category in the vector map.

[0070] The end-to-end autonomous driving model may also include a visualization module, which is used to intuitively display lane line information in the vector map.

[0071] As described above, by inputting the bird's-eye view features into the lane line construction model, a vector map including lane line categories output by the lane line construction model is obtained, thereby constructing the secondary categories of the vector map online, improving the recognition accuracy of the lane line type, and helping the end-to-end autonomous driving model to plan the vehicle trajectory more accurately.

[0072] In one embodiment, the lane line construction model may include: an improved map transformer (MapTransformer, MapTR) model.

[0073] The vector map perception module can pre-use a multilayer perceptron (MLP) to add a category classification head to the classification head of the initial map transformer model, so that the improved map transformer model can output the lane line category corresponding to the lane line feature.

[0074] As described above, by using a multi-layer perceptron, a category classification head is added to the classification head of the initial map transformer model to obtain an improved map transformer model, which can enhance the recognition ability of the map transformer model for lane line categories.

[0075] In one embodiment, during the data preprocessing stage, lane line features such as lane line location information, line color, and line type may be extracted and annotated from pre-prepared high-precision map data to obtain pre-processed high-precision map data.

[0076] Input the HD map data into the initial lane line construction model, and use the lane line information in the HD map data as labels to train the initial lane line construction model. During the training process, the initial lane line construction model will learn to extract lane line features from the input data and construct the lane line information in the vector map to obtain a trained lane line construction model.

[0077] The high-precision map data may include information such as the location information of the lane line, the line color, and the line type. The initial lane line construction model is a lane line construction model before training.

[0078] As described above, by inputting high-precision map data into the initial lane line construction model, using the lane line information in the high-precision map data as a label, the initial lane line construction model is trained to obtain a lane line construction model, so that the lane line construction model learns the precise lane line position information and lane line features, which can significantly improve the accuracy of identifying lane lines; and, using high-precision map data for training can enable the lane line construction model to learn broader and more complex lane line features, thereby enhancing the generalization ability of the lane line construction model in different environments.

[0079] In the above-mentioned embodiment, the online construction process and training process of the vector map of the lane line construction model are introduced. In the following embodiment, a more detailed description will be given on how to adjust the path points in the vehicle trajectory when the vehicle trajectory intersects the lane line, and it can be applied to any of the above embodiments.

[0080] The planning module in the end-to-end autonomous driving model may include: a post-processing module. The post-processing module is used to optimize the trajectory of the vehicle output by the planning module.

[0081] The post-processing module can determine whether the ego vehicle trajectory intersects with the lane line. If the ego vehicle trajectory intersects with the lane line, the post-processing module can calculate the second angle at the first intersection path point where the ego vehicle trajectory intersects with the lane line.

[0082] The second angle is the angle formed by the tangent direction of the vehicle trajectory at the first intersection path point and the tangent direction of the lane line. The post-processing module determines the optimization scheme of the vehicle trajectory according to the second angle.

[0083] In one embodiment, when the second angle at the first intersection path point is smaller than the preset offset angle, the degree of offset between the driving direction of the vehicle at the first intersection path point and the road is relatively low.

[0084] The post-processing module can determine the translation distance of each translation path point according to the second angle at the first intersection path point and the speed information of the vehicle. For example, the translation distance of the previous path point of the first intersection path point is 0.4 meters, the translation distance of the first intersection path point is 0.6 meters, and the translation distance of the next path point after the first intersection path point is 0.8 meters.

[0085] Based on the translation distance of each translation path point, each translation path point is adjusted to move the translation path point in parallel to the inner side of the road where the vehicle is located.

[0086] The preset offset angle is used to determine the offset degree of the vehicle's driving direction at the first intersection path point, which can be 10°, 15°, etc. The first intersection path point is the first path point in the vehicle's trajectory that intersects the lane line, and the speed information is the driving speed of the vehicle at its current location.

[0087] The translation path point is a path point to be adjusted in the vehicle trajectory, and the translation path point may include: the first intersection path point, the previous path point of the first intersection path point, and the path point after the first intersection path point.

[0088] See also Figure 2 , Figure 2 A schematic diagram of adjusting the trajectory of the ego vehicle by translation is shown. Exemplarily, the lane line includes a first lane line 21 and a second lane line 22. The planned ego vehicle trajectory (green trajectory) of the ego vehicle 20 intersects with the first lane line 21 at the first intersection path point 24, and the second angle p is the angle formed by the tangent direction of the ego vehicle trajectory at the first intersection path point 24 and the tangent direction of the first lane line 21.

[0089] When the second angle p is less than the preset offset angle, the post-processing module can determine the translation distances of the first intersection path point 24, the previous path point 23 of the first intersection path point, and the first path point 25 and the second path point 26 after the first intersection path point 24 based on the second angle p and the speed information of the vehicle 20.

[0090] Based on the translation distance of each translation path point, each translation path point is adjusted to move the translation path point in parallel to the inner side of the road where the vehicle is located.

[0091] As described above, when the second angle at the first intersection path point is less than the preset offset angle, the translation distance of the translation path point in the self-vehicle trajectory is determined according to the second angle and the driving speed of the self-vehicle, and based on the translation distance, each translation path point is moved to the inner side of the road where the self-vehicle is located. While ensuring that the degree of deviation of the driving direction of the self-vehicle at the first intersection path point is low, the translated self-vehicle trajectory complies with traffic regulations and at the same time makes the self-vehicle trajectory as smooth as possible.

[0092] In one embodiment, when the second angle at the first intersection path point is greater than or equal to the preset offset angle, the degree of deviation between the driving direction of the vehicle at the first intersection path point and the road is relatively high. In this scenario, even if the path points in the vehicle trajectory are translated, the excessively deviated vehicle trajectory cannot be corrected.

[0093] The post-processing module can delete the first intersection path point and the path points after the first intersection path point in the ego-vehicle trajectory to obtain a new ego-vehicle trajectory. The path in the new ego-vehicle trajectory is shortened, and the new ego-vehicle trajectory is divided using the total number of path points of the ego-vehicle trajectory before deletion. The travel time of the total number of path points remains unchanged, thereby obtaining the ego-vehicle trajectory after deceleration.

[0094] The total number of path points is the total number of path points of the vehicle trajectory, which can be set according to the actual business scenario.

[0095] See also Figure 3 , Figure 3 A schematic diagram of decelerating and adjusting the trajectory of the ego vehicle is shown. Exemplarily, the lane line includes a first lane line 21 and a second lane line 22. The planned ego vehicle trajectory (green trajectory) of the ego vehicle 20 intersects with the first lane line 21 at the first intersection path point 24, and the second angle q is the angle formed by the tangent direction of the ego vehicle trajectory at the first intersection path point 24 and the tangent direction of the first lane line 21.

[0096] When the second angle q is greater than or equal to the preset offset angle, the post-processing module may delete the first intersection path point 24 in the vehicle trajectory, and the first path point 25 and the second path point 26 after the first intersection path point 24 .

[0097] Since the path of the ego vehicle trajectory is shortened, the total number of path points of the ego vehicle trajectory before deletion is 6. The new ego vehicle trajectory is evenly divided into 6 parts. The travel time of the 6 path points in the ego vehicle trajectory remains unchanged, and the speed of the ego vehicle is reduced, thereby simulating the deceleration effect of the ego vehicle.

[0098] Understandably, Figure 3 The even division of the new ego vehicle trajectory shown is only an example, and other division methods may also be used as long as the simulation of ego vehicle deceleration can be achieved. The embodiment of the present application does not limit this.

[0099] As described above, by deleting the first intersection path point and the path points after the first intersection path point in the ego vehicle trajectory when the second angle of the first intersection path point is greater than or equal to the preset offset angle, a new ego vehicle trajectory is obtained. Since the ego vehicle trajectory is shortened, the new ego vehicle trajectory is evenly divided using the total number of path points of the ego vehicle trajectory before deletion. This can achieve a simulated deceleration and avoidance effect when the degree of deviation of the ego vehicle's driving direction at the first intersection path point is high, thereby ensuring that the vehicle can safely return to the correct lane.

[0100] To further introduce the trajectory planning process of autonomous driving, Figure 4 A flow chart of another trajectory planning method for autonomous driving is shown. Figure 4 As shown, the trajectory planning method may include the following steps:

[0101] Step 402: Obtain the bird's-eye view features corresponding to the current road image.

[0102] In this step, other modules in the end-to-end autonomous driving model can obtain the bird's-eye view features corresponding to the current road image, and can also obtain information such as the 3D detection frame of the target vehicle and the predicted trajectory of the target vehicle.

[0103] Among them, the target vehicle is a vehicle traveling in the area around the vehicle.

[0104] Step 404: Input the bird's-eye view features into the lane line construction model to obtain a vector map output by the lane line construction model.

[0105] In this step, the vector map perception module in the end-to-end autonomous driving model inputs the bird's-eye view features obtained from other modules into the lane line construction model to obtain a vector map including lane line categories output by the lane line construction model.

[0106] Step 406: If the initial ego-vehicle trajectory generated by the planning module intersects with the training lane line, the loss value of the loss function is calculated according to the number of point sets of the intersecting path points and the first angle.

[0107] In this step, the planning module in the end-to-end autonomous driving model can generate an initial vehicle trajectory. The training module in the end-to-end autonomous driving model determines the lane lines in the vector map according to the lane line categories in the vector map.

[0108] If the initial vehicle trajectory intersects with the training lane line in the vector map, the training module calculates the loss value of the loss function based on the number of points in the intersection path and the first angle.

[0109] Step 408: Based on the loss value, update the parameters of the planning module to obtain a trained end-to-end autonomous driving model.

[0110] In this step, the parameters of the planning module are updated based on the loss value. The training process of step 406 and step 408 is repeated to minimize the loss value and obtain a trained end-to-end autonomous driving model.

[0111] Step 410: Determine whether the second angle on the first intersection path point is smaller than a preset offset angle.

[0112] In this step, when the vehicle trajectory output by the trained planning module intersects with the lane line, the post-processing module can determine whether the second angle on the first intersection path point is less than the preset offset angle.

[0113] If the second angle at the first intersection path point is less than the preset offset angle, continue to execute step 412;

[0114] If the second angle at the first intersection path point is greater than or equal to the preset offset angle, step 416 is executed.

[0115] Step 412: Determine the translation distance of the translation path point according to the second angle and the speed information of the vehicle.

[0116] In this step, the post-processing module can determine the translation distance of each translation path point according to the second angle at the first intersection path point in the vehicle trajectory and the speed information of the vehicle.

[0117] Step 414: Based on the translation distance, the translation path point is moved parallel to the inner side of the road where the vehicle is located.

[0118] In this step, the post-processing module can move each translation path point in parallel to the inner side of the road where the vehicle is located based on the translation distance.

[0119] Step 416: Delete the first intersection path point and the path points after the first intersection path point in the ego vehicle trajectory to obtain a new ego vehicle trajectory.

[0120] In this step, the post-processing module can delete the first intersection path point and the path points after the first intersection path point in the ego vehicle trajectory to obtain a new ego vehicle trajectory.

[0121] Step 418: Use the total number of path points of the ego-vehicle trajectory before deletion to divide the new ego-vehicle trajectory.

[0122] In this step, the post-processing module can use the total number of path points of the ego-vehicle trajectory before deletion to divide the new ego-vehicle trajectory, shorten the path of the ego-vehicle trajectory, and keep the travel time of the ego-vehicle trajectory unchanged, simulating the deceleration of the ego-vehicle.

[0123] Figure 51 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. The electronic device may be, for example, a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a personal digital assistant, a server, a smart home appliance, a car computer, etc. Figure 5 At the hardware level, the electronic device includes a processor 502, an internal bus 504, a network interface 506, a memory 508, and a non-volatile memory 510, and may also include hardware required for other services. The processor 502 reads the corresponding computer program from the non-volatile memory 510 into the memory 508 and then runs it, forming an automatic driving trajectory planning device at the logical level. Of course, in addition to software implementations, this application does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0124] Figure 6 is a block diagram of a trajectory planning device for autonomous driving according to an exemplary embodiment of the present application. Figure 6 , the device may include: a lane line construction module 602 and a vehicle trajectory adjustment module 604, wherein:

[0125] The lane line construction module 602 is used to extract lane line features from the bird's-eye view features corresponding to the current road image, and construct lane line information in the vector map; wherein the current road image is an image of the road on which the autonomous driving vehicle is traveling;

[0126] The vehicle trajectory adjustment module 604 is used to adjust the path points in the vehicle trajectory based on the intersection point when the planned vehicle trajectory intersects with the lane line; the vehicle trajectory is the driving trajectory of the vehicle within a preset time in the future.

[0127] In one example, the lane line construction module 602, when used to extract lane line features from the bird's-eye view features corresponding to the current road image to construct lane line information in a vector map, includes: inputting the bird's-eye view features into a lane line construction model to obtain the vector map output by the lane line construction model; wherein the lane line construction model includes: a category classification head, and the category classification head is used to output the lane line category in the vector map.

[0128] In one example, the lane line construction model includes: an improved map-based transformer model; the lane line construction module 602, before being used to extract lane line features from the bird's-eye view features corresponding to the current road image and construct lane line information in the vector map, also includes: using the multi-layer perceptron to add a category classification head to the initial map-based transformer model classification head to obtain the improved map-based transformer model.

[0129] In one example, the lane line construction module 602, when used to train an end-to-end autonomous driving model, includes: if the initial self-vehicle trajectory intersects with the training lane line, obtaining the number of point sets and a first angle of the intersection path point; wherein the intersection path point is a path point in the initial self-vehicle trajectory that intersects with the training lane line, and the first angle is an angle formed by the tangent direction of the initial self-vehicle trajectory at the intersection path point and the tangent direction of the training lane line; according to the number of point sets and the first angle, calculating the loss value of the loss function; based on the loss value, updating the parameters of the planning module to obtain the trained end-to-end autonomous driving model.

[0130] In one example, the vehicle trajectory adjustment module 604, when used to adjust the path points in the vehicle trajectory based on the intersection point, includes: when the second angle at the first intersection path point in the vehicle trajectory is less than the preset offset angle, determining the translation distance of the translation path point according to the second angle and the speed information of the vehicle; wherein the second angle is the angle formed by the tangent direction of the vehicle trajectory at the first intersection path point and the tangent direction of the lane line, and the preset offset angle is used to determine the degree of deviation of the vehicle's driving direction at the first intersection path point; based on the translation distance, the translation path point is moved parallel to the inner side of the road where the vehicle is located.

[0131] In one example, the ego-vehicle trajectory adjustment module 604, when used to adjust the path points in the ego-vehicle trajectory based on the intersection point, includes: when the second angle on the first intersecting path point in the ego-vehicle trajectory is greater than or equal to the preset offset angle, deleting the first intersecting path point and the path points after the first intersecting path point to obtain a new ego-vehicle trajectory; and dividing the new ego-vehicle trajectory using the total number of path points of the ego-vehicle trajectory before deletion.

[0132] In one example, the lane line construction module 602, before being used to input the bird's-eye view features into the lane line construction model to obtain the vector map output by the lane line construction model, further includes: inputting high-precision map data into the initial lane line construction model, using the lane line information in the high-precision map data as a label, training the initial lane line construction model, and obtaining the lane line construction model.

[0133] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0134] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0135] In an exemplary embodiment, a non-temporary computer-readable storage medium including instructions is also provided, such as a memory including instructions. The above instructions can be executed by a processor of a trajectory planning device for autonomous driving to implement a method as described in any of the above embodiments.

[0136] The non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., and the present application does not limit this.

[0137] In an exemplary embodiment, a computer program product including a computer program / instruction is also provided. The computer program / instruction can be executed by a processor of a trajectory planning device for autonomous driving to implement any of the methods described in the above embodiments.

[0138] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0139] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the inventions claimed herein. The present application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0140] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A trajectory planning method for autonomous driving, characterized in that: The method is applied to an end-to-end autonomous driving model, and the method comprises: Extracting lane line features from the bird's-eye view features corresponding to the current road image to construct lane line information in the vector map; wherein the current road image is an image of the road on which the autonomous driving vehicle is traveling; When the planned vehicle trajectory intersects with the lane line, the path points in the vehicle trajectory are adjusted based on the intersection point; the vehicle trajectory is the driving trajectory of the vehicle within a preset time in the future.

2. The method according to claim 1, characterized in that The extracting lane line features from the bird's-eye view features corresponding to the current road image and constructing lane line information in the vector map includes: Inputting the bird's-eye view features into a lane line construction model to obtain the vector map output by the lane line construction model; The lane line construction model includes: a category classification head, and the category classification head is used to output the lane line category in the vector map.

3. The method according to claim 2, characterized in that The lane line construction model includes: an improved map transformer-based model; Before inputting the bird's-eye view features into the lane line construction model to obtain the vector map output by the lane line construction model, the method further includes: Using a multi-layer perceptron, the category classification head is added to the classification head of the initial map transformer-based model to obtain the improved map transformer-based model.

4. The method according to claim 1, characterized in that: The training process of the end-to-end autonomous driving model includes: If the initial ego vehicle trajectory intersects with the training lane line, the number of points of the intersection path point and the first angle are obtained; wherein the intersection path point is a path point in the initial ego vehicle trajectory that intersects with the training lane line, and the first angle is an angle formed by the tangent direction of the initial ego vehicle trajectory at the intersection path point and the tangent direction of the training lane line; Calculating a loss value of a loss function according to the number of point sets and the first angle; Based on the loss value, the parameters of the planning module are updated to obtain a trained end-to-end autonomous driving model.

5. The method according to claim 1, characterized in that: The adjusting the path point in the vehicle trajectory based on the intersection point includes: When the second angle at the first intersection path point in the self-vehicle trajectory is less than the preset offset angle, the translation distance of the translation path point is determined according to the second angle and the speed information of the self-vehicle; wherein the second angle is the angle formed by the tangent direction of the self-vehicle trajectory at the first intersection path point and the tangent direction of the lane line, and the preset offset angle is used to determine the degree of deviation of the driving direction of the self-vehicle at the first intersection path point; Based on the translation distance, the translation path point is moved parallel to the inner side of the road where the vehicle is located.

6. The method according to claim 1, characterized in that The adjusting the path point in the vehicle trajectory based on the intersection point includes: When the second angle at the first intersecting path point in the ego vehicle trajectory is greater than or equal to the preset offset angle, the first intersecting path point and the path points after the first intersecting path point are deleted to obtain a new ego vehicle trajectory; The new ego-vehicle trajectory is divided using the total number of path points of the ego-vehicle trajectory before deletion.

7. The method according to claim 2, characterized in that Before inputting the bird's-eye view features into the lane line construction model to obtain the vector map output by the lane line construction model, the method further includes: The high-precision map data is input into the initial lane line construction model, and the lane line information in the high-precision map data is used as a label to train the initial lane line construction model to obtain the lane line construction model.

8. A trajectory planning device for autonomous driving, characterized in that: The device is applied to an end-to-end autonomous driving model, and the device includes: A lane line construction module is used to extract lane line features from the bird's-eye view features corresponding to the current road image, and construct lane line information in the vector map; wherein the current road image is an image of the road on which the autonomous driving vehicle is traveling; The vehicle trajectory adjustment module is used to adjust the path points in the vehicle trajectory based on the intersection point when the planned vehicle trajectory intersects with the lane line; the vehicle trajectory is the driving trajectory of the vehicle within a preset time in the future.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 7 by running the executable instructions.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product having a computer program / instructions stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images