Driving path planning method and device, electronic equipment and readable storage medium
By fusing command, path, and driving state feature vectors and combining them with a bird's-eye view to generate a target driving path, the problem that end-to-end autonomous driving models cannot meet the complex commands of drivers is solved, and the route planning logic and safety of autonomous driving systems are improved.
Patent Information
- Application Number
- CN202410968282.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-22
AI Technical Summary
Existing end-to-end autonomous driving models lack human intervention interfaces, resulting in planned driving paths that cannot meet the driver's complex commands, such as overtaking and U-turns, and the output paths are relatively simplified.
By fusing the generated instruction feature vector, path description vector, and driving state feature vector, and combining them with the bird's-eye view feature vector, a target driving path is generated using weighted calculation and machine learning models, thereby optimizing the route planning of the autonomous driving system.
The generated driving paths better match the actual needs of drivers, improving the applicability and safety of the autonomous driving system.
Smart Images

Figure CN118991818A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a driving path planning method, device, electronic device and readable storage medium. Background Art
[0002] Autonomous driving technology has become one of the key research directions in the automotive industry. With the rapid development of computer vision, deep learning and sensor technology, autonomous vehicles have made significant progress in perception, decision-making and control. Traditional autonomous driving systems usually adopt a hierarchical structure, implementing perception, planning and control functions separately and then integrating them through a modular approach.
[0003] In recent years, end-to-end autonomous driving systems have begun to attract attention as an emerging technology. Unlike traditional hierarchical structures, end-to-end systems directly obtain data from sensors and output driving paths based on the acquired data to achieve driving functions, performing end-to-end learning and decision-making through a single unified model. The advantage of this approach is its simplicity and flexibility, which can better handle complex driving scenarios and unknown situations.
[0004] The existing end-to-end autonomous driving models at this stage lack an interface for human intervention in the driving path planned by the autonomous driving model, so that the driving path planned by the autonomous driving system can only conform to the perception results, that is, the planned driving path will not cause collisions, but cannot meet the driver's driving requirements. In addition, the driving paths planned and output by these models are relatively simple and cannot complete some complex instructions input by the driver, such as overtaking, U-turn, etc. Summary of the invention
[0005] The present application provides a driving path planning method, which is applied to an automatic driving system, and the method includes:
[0006] generating a command feature vector in response to a command input by a driver of the vehicle for indicating a driving path of the vehicle;
[0007] Acquire a path description vector; wherein the path description vector includes an initial driving path planned based on a bird's-eye view and a vector generated based on the initial driving path; the bird's-eye view includes environmental information of the vehicle's surroundings;
[0008] Acquire a driving state feature vector corresponding to the driving state information of the vehicle;
[0009] fusing the instruction feature vector, the path description vector, and the driving state feature vector to obtain a fusion vector;
[0010] The fusion vector and the feature vector corresponding to the bird's-eye view are weightedly calculated, and a target driving path for the automatic driving system is generated based on the weighted calculation result.
[0011] Optionally, performing weighted calculation on the fusion vector and the feature vector corresponding to the bird's-eye view, and generating a target driving path for the autonomous driving system based on the weighted calculation result, includes:
[0012] Performing weighted calculation on the fusion vector and the feature vector corresponding to the bird's-eye view to generate at least one two-dimensional coordinate point;
[0013] Based on the at least one two-dimensional coordinate point, a target driving path for the autonomous driving system is generated.
[0014] Optionally, performing weighted calculation on the fusion vector and the feature vector corresponding to the bird's-eye view, and generating a target driving path for the autonomous driving system based on the weighted calculation result, includes:
[0015] Inputting the fused vector and the bird's-eye view into a trained route planning model, so that the route planning model extracts a feature vector corresponding to the bird's-eye view, and based on an attention mechanism, calculating a weighted result of the fused vector and the feature vector corresponding to the bird's-eye view, to obtain a target vector including at least one two-dimensional coordinate point;
[0016] Based on at least one two-dimensional coordinate point in the target vector, a target driving path for the automatic driving system is generated.
[0017] Optionally, the method further includes:
[0018] Acquire a training data set for training the route planning model; the training data set includes at least one training sample annotated with a label; wherein the training sample includes instructions for indicating a vehicle driving path and scene data; the label includes a label for indicating a real driving path corresponding to the instruction;
[0019] The route planning model is supervisedly trained based on the training sample set until the route planning model converges.
[0020] Optionally, the route planning model is supervisedly trained based on the training sample set until the route planning model converges, including:
[0021] For any training sample in the training sample set, performing path mapping based on the scene data in the training sample to obtain an initial driving path, and extracting a path description vector of the initial driving path;
[0022] Based on the instructions in the training sample, generate an instruction feature vector;
[0023] Fusing the path description vector and the instruction feature vector to obtain a fusion vector;
[0024] Based on the attention mechanism, a weighted result of the fusion vector and the feature vector corresponding to the scene data is calculated to obtain a predicted target vector including at least one two-dimensional coordinate point;
[0025] generating a predicted driving path based on at least one two-dimensional coordinate point in the predicted target vector;
[0026] The predicted driving path is matched with the actual driving path in the label of the training sample, and the parameters of the route planning model are adjusted based on the matching result until the route planning model converges.
[0027] Optionally, the scene data includes road information and road dividing lines, and before matching the predicted driving path with the actual driving path in the label of the instruction, the method further includes:
[0028] Matching the predicted driving path with the road in the scene data to determine the road where the driving path is located in the scene data;
[0029] Based on a traffic road separation line loss function, calculating whether the driving path matches the road separation line in the scene data;
[0030] Based on the calculation result of the traffic road separation line loss function, the driving path is adjusted until the driving path matches the road separation line in the scene data.
[0031] Optionally, the fusing the instruction feature vector, the path description vector and the driving state feature vector to obtain a fusion vector includes:
[0032] Inputting the instruction feature vector, the path description vector, and the driving state feature vector into a multilayer perceptron, so that the multilayer perceptron concatenates the instruction feature vector, the path description vector, and the driving state feature vector to obtain a concatenated vector; and
[0033] The concatenated vector is reduced in dimension to obtain the fused vector.
[0034] The present application also provides a driving path planning device, which is applied to an automatic driving system, and the device includes:
[0035] an instruction feature vector generating unit, for generating an instruction feature vector in response to an instruction input by a driver of the vehicle for indicating a driving path of the vehicle;
[0036] A path description vector generating unit, used to obtain a path description vector corresponding to an initial driving path planned based on a bird's-eye view; wherein the bird's-eye view includes environmental information of the vehicle;
[0037] A driving state feature vector generating unit, used for acquiring a driving state feature vector corresponding to the driving state information of the vehicle;
[0038] A vector fusion unit, used for fusing the instruction feature vector, the path description vector and the driving state feature vector to obtain a fusion vector;
[0039] A path planning unit is used to perform path planning in the bird's-eye view based on the fusion vector to generate a target driving path for the automatic driving system.
[0040] The present application also provides an electronic device, comprising a communication interface, a processor, a memory and a bus, wherein the communication interface, the processor and the memory are interconnected via the bus;
[0041] The memory stores machine-readable instructions, and the processor executes the above method by calling the machine-readable instructions.
[0042] The present application also provides a computer-readable storage medium, which stores machine-readable instructions. When the machine-readable instructions are called and executed by a processor, the above method is implemented.
[0043] In the scheme recorded in the above embodiment, on the basis of the driving path for autonomous driving planned by the autonomous driving system based on the environmental information collected by the vehicle sensors, an additional instruction processing module is added, which combines the path description vector corresponding to the planned initial driving path with the instruction feature vector corresponding to the instruction input by the driver and the driving state feature vector corresponding to the vehicle driving state to obtain a fusion vector, and generates a driving path that better meets the actual needs of the driver based on the fusion vector, further optimizing the route planning logic of the autonomous driving system and improving the applicability and safety of the autonomous driving system. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 It is a flowchart of a driving route planning method provided by an exemplary embodiment.
[0045] Figure 2 It is a flowchart of a training process of a route planning model provided by an exemplary embodiment.
[0046] Figure 3 It is a schematic diagram of an automatic driving system planning a driving path provided by an exemplary embodiment.
[0047] Figure 4 It is a hardware structure diagram of an electronic device in which a driving route planning device is provided according to an exemplary embodiment.
[0048] Figure 5 It is a block diagram of a driving route planning device provided by an exemplary embodiment. DETAILED DESCRIPTION
[0049] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0050] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.
[0051] See also Figure 1 , Figure 1 This is a flow chart of a driving route planning method provided by an exemplary embodiment. The method may include the following steps:
[0052] Step 102 : generating an instruction feature vector in response to an instruction input by a driver of the vehicle for indicating a driving path of the vehicle.
[0053] In this specification, the instructions for indicating the vehicle's driving path may specifically include instructions such as going straight, turning left, turning right, stopping, overtaking, etc.
[0054] The specific method of obtaining the instruction input by the driver for indicating the driving path of the vehicle is not specifically limited in this specification. For example, the long text input by the driver can be obtained through the voice input module provided by the vehicle, such as "There is a car ahead on the current road, I should speed up to overtake now". The long text is further parsed through the language model to obtain the instruction contained in the long text, such as the instruction parsed by the language model from the long text as an overtaking instruction.
[0055] Sensors or cameras can also be used to collect driver behavior information, such as the steering angle of the steering wheel, the state of the brake or accelerator pedal, and light signals, and the collected information can be further interpreted as corresponding instructions. For example, if the steering wheel turns to the left more than a certain angle, it is interpreted as a left turn instruction.
[0056] After obtaining the instruction input by the driver for indicating the driving path of the vehicle, it is possible to extract features in the instruction in response to the instruction and generate an instruction feature vector corresponding to the instruction.
[0057] The specific method of generating the command feature vector corresponding to the command is not specifically limited in this specification. For example, a command feature vector can be designed to map the driver input to a feature vector space. For example, for a steering command, the command feature vector includes information such as the steering wheel rotation angle and the turn signal status.
[0058] For another example, a machine learning algorithm, such as a deep neural network, can be used to learn the features in the instruction from the data and generate a corresponding feature vector. Specifically, the instructions input by the driver and the corresponding instruction labels can be used as training data, so that the model can automatically learn and predict the features contained in the instruction, and generate the corresponding instruction feature vector based on the predicted features contained in the instruction.
[0059] For another example, an embedded method can be used to map the input instructions parsed from a long text to a low-dimensional vector space, thereby representing high-dimensional discrete data as dense continuous vectors. Specifically, a neural network model containing an embedding layer can be used, and the constructed deep neural network model can be trained using the prepared training data. During the training process, the parameters of the embedding layer will be learned so that the model can better map the input data to a continuous vector space. Through the embedding layer of the model, the instructions parsed from the long text can be mapped to a low-dimensional vector space, thereby converting the instructions input by the driver into the corresponding instruction feature vector.
[0060] Step 104, obtaining a path description vector; wherein the path description vector includes an initial driving path planned based on a bird's-eye view and a vector generated based on the initial driving path; the bird's-eye view contains environmental information of the vehicle's surroundings.
[0061] In this specification, sensors on the vehicle, such as cameras, laser radars, ultrasonic sensors, etc., can be used to collect data about the surrounding environment. Such data may include information such as obstacle locations, road signs, traffic signs, lane lines, etc.
[0062] Furthermore, a bird's-eye view can be constructed based on the vehicle's surrounding environment information. The bird's-eye view, also known as the BEV (Bird's Eye View) feature map, is an image or chart of the vehicle's surrounding environment built from above or at a high altitude based on the BEV Encoder, which is used to provide a global perspective and environmental perception. This perspective helps the autonomous driving system better understand the situation around the vehicle, including roads, traffic signs, obstacles, etc.
[0063] BEV Encoder is a neural network component used to build environmental perception, which is usually used in autonomous driving systems. The role of BEV Encoder is to build a bird's-eye view based on the environmental information around the vehicle, usually data from sensors, and further use it for tasks such as target detection, path planning, and driving trajectory prediction.
[0064] In this specification, the autonomous driving system can plan the driving path based on the constructed bird's-eye view. Specifically, the motion pre-processing module in the autonomous driving system can plan the driving path in combination with the high-precision map pre-processing module, the tracking pre-processing module, and the bird's-eye view constructed by BEV Encode.
[0065] The high-precision map pre-processing module is responsible for creating and updating high-precision maps. These maps contain detailed geographic information, such as road layouts, traffic signs, building locations, etc. The tracking pre-processing module is responsible for tracking moving targets, such as vehicles and pedestrians. The tracking pre-processing module can use historical data to predict the future location and behavior of the target.
[0066] The motion pre-processing module can combine the data in the high-precision map pre-processing module, the tracking pre-processing module, and the bird's-eye view map constructed by BEVEncode to generate a path description vector for describing the initial driving path of the vehicle.
[0067] Specifically, the autonomous driving system can use image processing technology and computer vision algorithms to extract key elements such as roads, pedestrians, other vehicles, traffic signs, obstacles, etc. from the bird's-eye view. For each key element identified, such as lane lines and obstacles, the system will generate a corresponding feature vector containing information such as location coordinates, width, and obstacle distance.
[0068] The feature vector corresponding to the extracted bird's-eye view features can be further matched with the high-precision map data on the vehicle to confirm the current location of the vehicle and obtain the surrounding road network structure. Then, the A* algorithm, Dijkstra algorithm or more advanced graph search algorithm can be used to plan the initial driving path from the current location to the target location on the map.
[0069] After obtaining the initial driving path planned by the automatic driving system, a path description vector describing the initial driving path of the vehicle can be further generated based on the initial driving path.
[0070] In one implementation, the path description vector may specifically include an ego-vehicle trajectory query vector and an ego-vehicle tracking query vector.
[0071] The ego vehicle trajectory query vector describes the expected driving path of the vehicle over a period of time. This vector usually contains the following information: start time and end time, current position and speed of the ego vehicle, trajectory preference, trajectory constraints, prediction time step, etc.
[0072] Among them, the start time and end time are used to describe the time range of trajectory prediction; the current position and speed of the vehicle are used to describe the current position and speed of the autonomous driving vehicle; the trajectory preference is used to describe the preference of the vehicle trajectory, such as the shortest path, the fastest path, etc.; the trajectory constraint is used to describe the constraint conditions of the vehicle trajectory, such as the maximum acceleration, the maximum turning angle, etc. The prediction time step is used to describe the time step of trajectory prediction, etc.
[0073] The ego-vehicle trajectory query vector is a vector output by the autonomous driving system that is used to predict the vehicle's future driving trajectory in order to perform tasks such as path planning, obstacle avoidance, and decision-making.
[0074] The self-driving vehicle tracking query vector describes the tracking of the surrounding targets by the autonomous vehicle. This vector usually contains the following information: the position and speed of the tracking target, the characteristics of the tracking target, and the tracking algorithm parameters.
[0075] The position and speed of the tracking target are used to describe the current position and speed of the target to be tracked by the autonomous vehicle. The characteristics of the tracking target are used to describe the characteristic information of the tracking target, such as the type and size of the target. The tracking algorithm parameters are used to describe the algorithm parameters used to track the target, such as the parameters of the Kalman filter.
[0076] The ego-vehicle tracking query vector is used to help the autonomous vehicle track surrounding objects to achieve safe driving and avoid potential collisions.
[0077] Step 106: Acquire a driving state feature vector corresponding to the driving state information of the vehicle.
[0078] In this specification, the driving status information of the vehicle can be specifically collected from various sensors and control systems of the vehicle via a CAN bus (Controller Area Network).
[0079] Among them, the vehicle's driving status characteristics may specifically include the vehicle's historical trajectory, the vehicle's navigation instructions, and the vehicle's high-level characteristics. In addition, the vehicle's driving status characteristics may also include other content such as vehicle posture information, vehicle braking system status, vehicle suspension system information, etc., which are not specifically limited in this specification.
[0080] The vehicle's historical trajectory records the vehicle's driving path over the past period of time. By analyzing the historical trajectory, the autonomous driving system can understand the vehicle's driving pattern, such as whether there are sharp turns.
[0081] The navigation instructions of the vehicle include instructions such as turn left, turn right, and go straight, and these instructions usually come from the vehicle's navigation system or a preset driving plan.
[0082] The high-level features of the ego vehicle can usually include the speed and acceleration along the x direction, the speed and acceleration along the y direction, the angular velocity of the ego vehicle in the current frame, etc. These parameters reflect the dynamic state of the vehicle and are crucial for controlling acceleration, deceleration, maintaining a safe distance, and other operations.
[0083] When planning a driving route, integrating these rich vehicle status features can not only improve the decision-making accuracy and driving safety of the autonomous driving system, but also promote vehicle health management and improve the overall driving experience and efficiency.
[0084] After the driving state characteristics of the vehicle are obtained, a corresponding driving state characteristic vector may be generated. The specific method of generating the driving state characteristic vector is not specifically limited in this specification.
[0085] For example, a machine learning algorithm can be used to learn driving state features from the driving state information of the vehicle and generate a corresponding feature vector. Specifically, the driving state information of the vehicle and the corresponding marked driving state features can be used as training data to allow the model to automatically learn the driving state features contained in the driving state information and combine them into a driving state feature vector based on the learned driving state features.
[0086] Step 108: Fusing the instruction feature vector, the path description vector, and the driving state feature vector to obtain a fusion vector.
[0087] In this specification, after obtaining the instruction feature vector, the path description vector and the driving state feature vector, these three vectors can be fused to obtain a fused vector, and the driving path can be further planned based on the fused vector.
[0088] The specific method of fusing the three vectors is not specifically limited in this specification. For example, the three vectors can be directly concatenated to obtain a fused vector, or a weight can be assigned to each vector, and the weighted sum of each vector and its weight can be calculated to obtain a fused vector.
[0089] Since each vector is obtained through different models or methods, it is also possible to consider using ensemble learning methods, such as voting, stacking, or weighted averaging, to fuse the vectors output by multiple models to obtain a fused vector.
[0090] In addition, a multi-layer perceptron may be used to concatenate the above-mentioned instruction feature vector, path description vector, and driving status feature vector, and perform dimensionality reduction on the concatenated vector to obtain a fusion vector.
[0091] Multi-Layer Perceptron (MLP) is a neural network model. It consists of one or more fully connected hidden layers and an output layer. Each hidden layer contains multiple neurons, and each neuron is connected to all neurons in the previous layer. MLP learns complex patterns and feature representations of input data by performing a series of nonlinear transformations on the input data.
[0092] In this specification, a multi-layer perceptron may be used to achieve fusion of multiple vectors into one vector. First, after obtaining the instruction feature vector, the path description vector, and the driving state feature vector, it is necessary to determine that these vectors have the same length or dimension so that they can be input into the MLP model.
[0093] The structure of the MLP model can be further determined, including the number and size of hidden layers, the choice of activation function, etc. The size of the input layer of the MLP model should match the length of the vector to be fused, and the size of the output layer can be determined according to the specific task requirements.
[0094] The MLP model can then be trained using the prepared training data. During the training process, the model will learn how to concatenate the input vectors and perform dimensionality reduction on the concatenated vectors.
[0095] After the model training and evaluation are completed, the trained MLP model can be applied to the instruction feature vector, the path description vector and the driving state feature vector to obtain a fusion vector.
[0096] The MLP model can learn more complex feature representations, so that the MLP model can better capture the complex relationship between input vectors, thereby improving the representation ability of the fused vector.
[0097] Step 110, performing weighted calculation on the fusion vector and the feature vector corresponding to the bird's-eye view, and generating a target driving path for the autonomous driving system based on the weighted calculation result.
[0098] In this specification, after obtaining the fusion vector, environmental features can be extracted from the bird's-eye view and the corresponding feature vector can be generated. The autonomous driving system can use image processing technology and computer vision algorithms to extract key elements such as roads, pedestrians, other vehicles, traffic signs, obstacles, etc. from the bird's-eye view. For each identified key element such as lane lines and obstacles, the system will generate a corresponding feature vector. This process ensures that the visual information in the bird's-eye view can be encoded in a machine-understandable way.
[0099] The fusion vector and the feature vector corresponding to the bird's-eye view can further be weighted. The purpose of weighted calculation is to balance the importance of the fusion vector and the feature vector corresponding to the bird's-eye view in the path planning process. Different application scenarios may require different weights to different factors. For example, in a crowded urban environment, the weight of the feature vector corresponding to the bird's-eye view may be higher; while on an open highway, more emphasis may be placed on driving status and user instruction characteristics. The specific weight distribution may be based on expert knowledge, historical data analysis, or obtained through automatic optimization by a machine learning algorithm, and is not specifically limited in this specification. During calculation, the corresponding elements of the two sets of vectors are multiplied by their weights, and then summed to form a new decision vector.
[0100] Finally, the decision vector obtained by the above weighted calculation can be used to search for the most suitable target driving path in the possible driving path space through path planning algorithms such as A* algorithm, RRT fast random tree algorithm, etc. This path is then output to the autonomous driving system and can be used as a basis for actual driving.
[0101] In one embodiment, the driving path is composed of at least one two-dimensional coordinate point, and a driving path can be represented as a series of coordinate point sequences. In a sequence, each coordinate pair represents a position point on the path, and these points are connected together to form a complete path for the vehicle to travel.
[0102] In this specification, after obtaining the fusion vector, the fusion vector and the feature vector corresponding to the bird's-eye view can be weighted calculated, and a path planning algorithm can be further used in the bird's-eye view to obtain at least one two-dimensional coordinate point, and a coordinate point sequence can be generated, and the target driving path can be generated based on this coordinate point sequence.
[0103] In one embodiment, a trained machine learning model may be used in combination with an attention mechanism to predict at least one coordinate point based on a bird's-eye view and an additional fusion vector to generate a target driving path.
[0104] Specifically, the fusion vector and the feature vector corresponding to the bird's-eye view can be input into a trained route planning model, so that the route planning model can calculate the weighted result of the fusion vector and the feature vector corresponding to the bird's-eye view based on the attention mechanism to obtain a target vector containing at least one two-dimensional coordinate point.
[0105] In this specification, for the route planning model, an attention layer can be designed to dynamically assign weights to different parts of the fusion vector and the BEV feature vector. The fusion vector and the BEV feature vector can be further weighted fused through the attention mechanism. Finally, the fused features are input to the decoder part, which is responsible for predicting a target vector containing a series of two-dimensional coordinate points.
[0106] After obtaining a series of two-dimensional coordinate points predicted by the model, a target driving path for the autonomous driving system can be generated based on at least one two-dimensional coordinate point in the target vector.
[0107] In one embodiment, an extended training data set may be used to train the route planning model. The extended training data set includes at least one training sample with a label; the training sample includes instructions for indicating a vehicle driving path and scene data; the label includes a label for indicating a real driving path corresponding to the instruction;
[0108] In traditional data sets, training samples usually only include scene data, where scene data can specifically be data associated with each scene, including the vehicle's movement pattern, road type, weather conditions, and rich information about the surrounding environment collected by vehicle sensors, such as roads, vehicles, pedestrians, etc.
[0109] In this specification, when training the route planning model, in addition to using scene data, labeled instructions for indicating the vehicle's driving path are specifically introduced, so that the driving path predicted by the route planning model is more in line with the driver's needs.
[0110] After the training data set is input into the route planning model, supervised training is performed on the route planning model based on the training sample set until the route planning model converges.
[0111] See also Figure 2 , Figure 2The present invention is a flowchart of a training process of a route planning model provided by an exemplary embodiment. The method may include the following steps:
[0112] Step 202: for any training sample in the training sample set, perform path mapping based on the scene data in the training sample to obtain an initial driving path, and extract a path description vector of the initial driving path.
[0113] In this specification, the route planning model can use the scene data and scene information in the training data set to construct a road network diagram, including information such as roads, intersections, and lane lines.
[0114] Furthermore, according to the scene information, a path planning algorithm can be used to search for the optimal path in the scene data to reach the target location.
[0115] The generated driving path can be further converted into a path description vector, wherein the path description vector can specifically include the above-mentioned vehicle trajectory query vector and vehicle tracking query vector, which will not be described in detail here.
[0116] Step 204: Generate an instruction feature vector based on the instructions in the training sample.
[0117] In this specification, the training data set may include at least one instruction for indicating a vehicle's driving path, such as instructions for going straight, turning left, turning right, stopping, overtaking, and the like.
[0118] The specific method of generating the instruction feature vector corresponding to the instruction can use a machine learning algorithm to learn the features in the instruction from the data and generate a corresponding feature vector; or use an embedded method to map the input instruction parsed from a long text to a low-dimensional vector space to generate a feature vector, which will not be repeated here.
[0119] In step 206, the path description vector and the instruction feature vector are fused to obtain a fused vector.
[0120] In this specification, during training, only the path description vector and the instruction feature vector may be fused to obtain a fused vector.
[0121] The two vectors can be directly concatenated to obtain a fusion vector, or a weight can be assigned to each vector, and the weighted sum of each vector and its weight can be calculated to obtain a fusion vector. A multi-layer perceptron can also be used to concatenate the instruction feature vector, the path description vector, and the driving state feature vector, and the concatenated vector can be reduced in dimension to obtain a fusion vector.
[0122] Step 208, based on the attention mechanism, calculate the weighted result of the fusion vector and the feature vector corresponding to the scene data to obtain a predicted target vector containing at least one two-dimensional coordinate point.
[0123] For the fused vector, the fused vector and the feature vector corresponding to the scene data can be input into the attention layer, and the attention mechanism can be used to dynamically assign weights to different parts of the fused vector and the feature vector corresponding to the scene data. The fused vector and the feature vector corresponding to the scene data are further weighted fused through the attention mechanism, and finally the fused vector is input into the decoder part, which is responsible for predicting a target vector containing a series of two-dimensional coordinate points.
[0124] Step 210: Generate a predicted driving path based on at least one two-dimensional coordinate point in the predicted target vector.
[0125] Since the driving path can be composed of at least one two-dimensional coordinate point, a driving path can be represented as a series of coordinate point sequences. In a sequence, each coordinate pair represents a position point on the path, and these points are connected together to form the complete path of the vehicle.
[0126] Therefore, after obtaining a target vector including a series of two-dimensional coordinate points, a driving path may be generated based on at least one two-dimensional coordinate point in the target vector.
[0127] Step 212, matching the predicted driving path with the actual driving path in the label of the training sample, and adjusting the parameters of the route planning model based on the matching result until the route planning model converges.
[0128] After the route planning model outputs the driving path, the model-predicted driving path can be matched with the actual driving path in the label of the training sample. The route planning model can predict the driving path based on the training data set, and further the route planning model can determine the difference between the predicted driving path and the actual driving path in the label of the instruction, and adjust the parameters of the route planning model based on the matching result until the route planning model converges.
[0129] Specifically, each point on the predicted path can be matched with the nearest neighbor point on the actual path to calculate the distance or error between the two. Alternatively, evaluation indicators such as Average Displacement Error (ADE), Final Displacement Error (FDE), or trajectory overlap rate can be used to quantify the difference between the predicted driving path and the actual driving path.
[0130] Furthermore, by analyzing the matching results, we can identify the scenarios or conditions where the model performs poorly, such as specific road types, turning radius, traffic density, etc. Based on the evaluation results, we adjust the model's hyperparameters, such as learning rate, regularization parameters, and weight distribution in the attention mechanism, in order to reduce the prediction error. We further retrain the model using the adjusted parameters until the route planning model meets the requirements.
[0131] In one embodiment, the driving path output by the route planning model can also be matched with the scene data to determine the road on which the driving path is located in the scene data, and a traffic road separation line loss function can be further used to determine whether the driving path matches the road separation lines in the scene data, and then based on the calculation result of the traffic road separation line loss function, the driving path is adjusted until the driving path matches the road separation lines in the scene data.
[0132] The specific form of the traffic dividing line loss function is not specifically limited in this specification, for example, it can be a cross entropy loss function, a Dice loss function, a boundary loss function, etc.
[0133] Based on the value of the loss function, it is possible to determine what adjustments need to be made to the model output trajectory to better match the scene data and road dividers. The goal of the adjustment is to make the trajectory output by the model more consistent with the road information and traffic rules in the scene data to reduce the value of the loss function. The above process is iterated until the matching degree between the model output trajectory and the scene data reaches a satisfactory level.
[0134] Therefore, scene data and loss functions can be used to guide model training to improve the model's understanding of road rules and structures, so that the driving path output by the model is more in line with road rules.
[0135] See also Figure 3 , Figure 3 It is a schematic diagram showing an exemplary embodiment of an automatic driving system planning a driving path.
[0136] like Figure 3 As shown, sensors on the vehicle, such as cameras, lidar, ultrasonic sensors, etc., can be used to collect data about the surrounding environment. Such data may include information such as obstacle locations, road signs, traffic signs, lane lines, etc. Furthermore, a BEV Encoder can be used to construct a BEV bird's-eye view based on the vehicle's surrounding environment information.
[0137] The motion pre-processing module in the autonomous driving system can plan the initial driving path in combination with the high-precision map pre-processing module, the tracking pre-processing module, and the bird's-eye view constructed by BEV Encode, and obtain a path description vector corresponding to the initial driving path. The path description vector may include a self-vehicle trajectory query vector and a self-vehicle tracking query vector.
[0138] The automatic driving system can collect information from various vehicle sensors and control systems through the CAN bus, obtain the vehicle's driving state information, extract driving state characteristics, and generate a driving state feature vector corresponding to the driving state characteristics.
[0139] The automatic driving system can also obtain instructions input by the driver to indicate the vehicle's driving path, and generate an instruction feature vector corresponding to the instructions.
[0140] Among them, the specific method of obtaining the instructions input by the driver to indicate the vehicle's driving path can be to obtain the long text input by the driver through the voice input module provided by the vehicle; or to collect the driver's behavior information by using sensors or cameras.
[0141] Furthermore, the path description vector, the instruction feature vector and the driving state feature vector can be fused through a multi-layer perceptron to obtain a fusion vector.
[0142] Finally, the pre-trained path planning model can be used to plan a driving path based on the fusion vector and the BEV bird's-eye view that will not collide with other vehicles and obstacles and conforms to the driver's intention and kinematic laws.
[0143] In the scheme recorded in the above embodiment, on the basis of the driving path for autonomous driving planned by the autonomous driving system based on the environmental information collected by the vehicle sensors, an additional instruction processing module is added, which combines the path description vector corresponding to the planned initial driving path with the instruction feature vector corresponding to the instruction input by the driver and the driving state feature vector corresponding to the vehicle driving state to obtain a fusion vector, and generates a driving path that better meets the actual needs of the driver based on the fusion vector, further optimizing the route planning logic of the autonomous driving system and improving the applicability and safety of the autonomous driving system.
[0144] See also Figure 4 , Figure 4This is a hardware structure diagram of an electronic device in which a driving route planning device is located, which is shown as an exemplary embodiment. At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410, and of course may also include hardware required for other services. One or more embodiments of this specification can be implemented based on software, such as the processor 402 reading the corresponding computer program from the non-volatile memory 410 into the memory 408 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0145] See also Figure 5 , Figure 5 is a block diagram of a driving path planning device shown in an exemplary embodiment. The driving path planning device can be applied to Figure 3 In the electronic device shown in the figure, the technical solution of this specification is implemented. Among them, the above-mentioned driving path planning device may include:
[0146] The instruction feature vector generating unit 502 is used to generate an instruction text feature vector in response to an instruction input by the driver of the vehicle for indicating a driving path of the vehicle;
[0147] A path description vector generating unit 504 is used to obtain a path description vector; wherein the path description vector includes an initial driving path planned based on a bird's-eye view and a vector generated based on the initial driving path; the bird's-eye view includes environmental information of the vehicle's surroundings;
[0148] A driving state feature vector generating unit 506, configured to obtain a driving state feature vector corresponding to the driving state information of the vehicle;
[0149] A vector fusion unit 508, configured to fuse the instruction feature vector, the path description vector and the driving state feature vector to obtain a fusion vector;
[0150] The path planning unit 510 is used to perform weighted calculation on the fusion vector and the feature vector corresponding to the bird's-eye view, and generate a target driving path for the automatic driving system based on the weighted calculation result.
[0151] Performing weighted calculation on the fusion vector and the feature vector corresponding to the bird's-eye view to generate at least one two-dimensional coordinate point;
[0152] Based on the at least one two-dimensional coordinate point, a target driving path for the autonomous driving system is generated.
[0153] In this embodiment, the path planning unit is further used to
[0154] Inputting the fused vector and the bird's-eye view into a trained route planning model, so that the route planning model extracts a feature vector corresponding to the bird's-eye view, and based on an attention mechanism, calculating a weighted result of the fused vector and the feature vector corresponding to the bird's-eye view, to obtain a target vector including at least one two-dimensional coordinate point;
[0155] Based on at least one two-dimensional coordinate point in the target vector, a target driving path for the automatic driving system is generated.
[0156] In this embodiment, the device further includes:
[0157] A path planning model training unit, for performing path planning based on scene data in any training sample in the training sample set to obtain an initial driving path, and extracting a path description vector of the initial driving path;
[0158] Based on the instructions in the training sample, generate an instruction feature vector;
[0159] Fusion of the path description vector and the instruction feature vector to obtain a fusion vector;
[0160] Based on the attention mechanism, a weighted result of the fusion vector and the feature vector corresponding to the scene data is calculated to obtain a predicted target vector including at least one two-dimensional coordinate point;
[0161] generating a predicted driving path based on at least one two-dimensional coordinate point in the predicted target vector;
[0162] The predicted driving path is matched with the actual driving path in the label of the training sample, and the parameters of the route planning model are adjusted based on the matching result until the route planning model converges.
[0163] In this embodiment, the scene data also includes road information and road dividing lines, and the device also includes:
[0164] A road dividing line matching unit, used to match the predicted driving path with the road in the scene data to determine the road where the driving path is located in the scene data;
[0165] Based on a traffic road separation line loss function, calculating whether the driving path matches the road separation line in the scene data;
[0166] Based on the calculation result of the traffic road separation line loss function, the driving path is adjusted until the driving path matches the road separation line in the scene data.
[0167] In this embodiment, the vector fusion unit is further used to input the instruction feature vector, the path description vector and the driving state feature vector into a multilayer perceptron, so that the multilayer perceptron splices the instruction feature vector, the path description vector and the driving state feature vector to obtain a spliced vector; and
[0168] The concatenated vector is reduced in dimension to obtain the fused vector.
[0169] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.
[0170] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Ordinary technicians in this field can understand and implement it without paying creative work.
[0171] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0172] This specification also provides an embodiment of a computer-readable storage medium. The computer-readable storage medium stores machine-readable instructions, and when the machine-readable instructions are called and executed by a processor, the driving path planning method provided in any embodiment of this specification can be implemented.
[0173] The computer-readable storage medium provided in the embodiments of this specification may specifically include, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical fiber card. That is, the readable storage medium includes a readable medium that can store or transmit information.
[0174] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.
[0175] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0176] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0177] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.
[0178] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0179] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0180] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "one", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0181] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0182] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.
Claims
1. A driving path planning method, applied to an automatic driving system of a vehicle, the method comprising: generating a command feature vector in response to a command input by a driver of the vehicle for indicating a driving path of the vehicle; Acquire a path description vector; wherein the path description vector includes an initial driving path planned based on a bird's-eye view and a vector generated based on the initial driving path; the bird's-eye view includes environmental information of the vehicle's surroundings; Acquire a driving state feature vector corresponding to the driving state information of the vehicle; fusing the instruction feature vector, the path description vector, and the driving state feature vector to obtain a fusion vector; The fusion vector and the feature vector corresponding to the bird's-eye view are weightedly calculated, and a target driving path for the automatic driving system is generated based on the weighted calculation result.
2. The method according to claim 1, wherein the fusion vector and the feature vector corresponding to the bird's-eye view are weighted and a target driving path for the autonomous driving system is generated based on the weighted calculation result, comprising: Performing weighted calculation on the fusion vector and the feature vector corresponding to the bird's-eye view to generate at least one two-dimensional coordinate point; Based on the at least one two-dimensional coordinate point, a target driving path for the autonomous driving system is generated.
3. The method according to claim 1, wherein the fusion vector and the feature vector corresponding to the bird's-eye view are weighted and a target driving path for the autonomous driving system is generated based on the weighted calculation result, comprising: Inputting the fused vector and the bird's-eye view into a trained route planning model, so that the route planning model extracts a feature vector corresponding to the bird's-eye view, and based on an attention mechanism, calculating a weighted result of the fused vector and the feature vector corresponding to the bird's-eye view, to obtain a target vector including at least one two-dimensional coordinate point; Based on at least one two-dimensional coordinate point in the target vector, a target driving path for the automatic driving system is generated.
4. The method according to claim 3, further comprising: Acquire a training data set for training the route planning model; The training data set includes at least one training sample annotated with a label; wherein the training sample includes instructions for indicating a vehicle driving path and scene data; the label includes a label for indicating a real driving path corresponding to the instruction; The route planning model is supervisedly trained based on the training sample set until the route planning model converges.
5. The method according to claim 4, performing supervised training on the route planning model based on the training sample set until the route planning model converges, comprising: For any training sample in the training sample set, performing path mapping based on the scene data in the training sample to obtain an initial driving path, and extracting a path description vector of the initial driving path; Based on the instructions in the training sample, generate an instruction feature vector; Fusing the path description vector and the instruction feature vector to obtain a fusion vector; Based on the attention mechanism, a weighted result of the fusion vector and the feature vector corresponding to the scene data is calculated to obtain a predicted target vector including at least one two-dimensional coordinate point; generating a predicted driving path based on at least one two-dimensional coordinate point in the predicted target vector; The predicted driving path is matched with the actual driving path in the label of the training sample, and the parameters of the route planning model are adjusted based on the matching result until the route planning model converges.
6. The method according to claim 5, wherein the scene data includes road information and road dividing lines, and before matching the predicted driving path with the actual driving path in the label of the instruction, the method further includes: Matching the predicted driving path with the road in the scene data to determine the road where the driving path is located in the scene data; Based on a traffic road separation line loss function, calculating whether the driving path matches the road separation line in the scene data; Based on the calculation result of the traffic road separation line loss function, the driving path is adjusted until the driving path matches the road separation line in the scene data.
7. The method according to claim 1, wherein the fusing the instruction feature vector, the path description vector and the driving state feature vector to obtain a fusion vector comprises: Inputting the instruction feature vector, the path description vector, and the driving state feature vector into a multilayer perceptron, so that the multilayer perceptron concatenates the instruction feature vector, the path description vector, and the driving state feature vector to obtain a concatenated vector; and The concatenated vector is reduced in dimension to obtain the fused vector.
8. A driving path planning device, applied to an automatic driving system of a vehicle, the device comprising: An instruction feature vector generating unit, configured to generate an instruction text feature vector in response to an instruction input by a driver of the vehicle for indicating a driving path of the vehicle; A path description vector generating unit, used for obtaining a path description vector; wherein the path description vector includes an initial driving path planned based on a bird's-eye view and a vector generated based on the initial driving path; the bird's-eye view includes environmental information of the vehicle's surroundings; A driving state feature vector generating unit, used for acquiring a driving state feature vector corresponding to the driving state information of the vehicle; A vector fusion unit, used for fusing the instruction feature vector, the path description vector and the driving state feature vector to obtain a fusion vector; A path planning unit is used to perform weighted calculation on the fusion vector and the feature vector corresponding to the bird's-eye view, and generate a target driving path for the automatic driving system based on the weighted calculation result.
9. An electronic device, comprising a communication interface, a processor, a memory and a bus, wherein the communication interface, the processor and the memory are interconnected via the bus; The memory stores machine-readable instructions, and the processor executes the method according to any one of claims 1 to 7 by calling the machine-readable instructions.
10. A computer-readable storage medium storing machine-readable instructions, wherein the machine-readable instructions, when called and executed by a processor, implement the method according to any one of claims 1 to 7.
Citation Information
Cited By
Driving path planning method and device, electronic equipment and storage medium
CN120406456A
Training method of trajectory generation model, driving planning method and vehicle
CN122493196A
Method for ascertaining potential trajectories when controlling a robot device
US20260035020A1
