Automatic driving reference line generation method and device, equipment, medium and vehicle

Abstract

The invention relates to a method, a device and equipment for generating an automatic driving reference line, a medium and a vehicle. The method comprises the following steps: acquiring navigation information and an SD map of an automatic driving vehicle; extracting spatial feature data based on the SD map; obtaining time sequence characteristic data based on the navigation information; fusing the spatial feature data and the time sequence feature data to obtain fused feature data; and generating a target reference line of the autonomous vehicle according to the fused feature data. According to the embodiment of the invention, the end-to-end reference line can be generated in combination with the navigation information and the SD map, and the reference line can be reasonably and accurately planned.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to a method for generating autonomous driving reference lines, an apparatus for generating autonomous driving reference lines, an electronic device, a computer-readable storage medium, and a vehicle. Background Technology

[0002] Currently, the main solutions for generating reference lines for end-to-end autonomous driving include end-to-end learning methods based on pure vision, methods based on traditional path planning algorithms, and hybrid methods.

[0003] While purely vision-based end-to-end learning methods can achieve a direct mapping from perception to control, they are prone to insufficient generalization ability in complex scenarios. These methods typically only utilize sensor data such as cameras or LiDAR, lacking effective use of prior map information, resulting in performance degradation in unfamiliar scenes.

[0004] Traditional reference line generation methods based on geometry programming, while offering good interpretability and stability, struggle to adapt to dynamically changing traffic environments and suffer from high computational complexity. These methods typically require accurate vehicle state information and environmental perception results, and are prone to producing unreasonable planning outcomes when sensor noise or perception errors occur. Summary of the Invention

[0005] One objective of this invention is to provide a method for generating reference lines for autonomous driving, so as to solve the problem of unreasonable reference line planning in the prior art; a second objective is to provide an apparatus for generating reference lines for autonomous driving; a third objective is to provide an electronic device; a fourth objective is to provide a computer-readable storage medium; and a fifth objective is to provide a vehicle.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for generating reference lines for autonomous driving, the method comprising: Obtain navigation information and SD maps from autonomous vehicles; Spatial feature data is extracted based on the SD map; Temporal feature data is obtained based on the navigation information; The spatial feature data and the temporal feature data are fused to obtain fused feature data; The target reference line for the autonomous vehicle is generated based on the fused feature data.

[0007] Optionally, generating the target reference line for the autonomous vehicle based on the fused features includes: A first candidate reference line is generated based on the fusion features; The first candidate reference line is optimized according to the preset optimization objective function to obtain the second candidate reference line; The second candidate reference line is smoothed to obtain the target reference line of the autonomous vehicle.

[0008] Optionally, generating the first candidate reference line based on the fusion features includes: The fused features are input into a preset multilayer perceptron, which outputs multiple candidate points corresponding to the fused features; the multilayer perceptron is used to output corresponding candidate points based on the input fused features. A first candidate reference line is generated based on the candidate points.

[0009] Optionally, optimizing the first candidate reference line according to a preset optimization objective function to obtain the second candidate reference line includes: Create the kinematic model and reference line constraints for the autonomous vehicle; Based on the kinematic model and the constraints, determine the total cost data corresponding to the preset optimization objective function; A second candidate reference line is determined from the first candidate reference line based on the total cost data; The second candidate reference line is obtained by optimizing the first candidate parameter line.

[0010] Optionally, determining the total cost data corresponding to the preset optimization objective function based on the kinematic model and the constraints includes: Based on the kinematic model and the constraints, the curvature information, obstacle information, velocity information, and acceleration information of the first candidate reference line are determined. The smoothness cost data of the first candidate reference line is determined based on the curvature information; The safety cost data of the first candidate reference line is determined based on the obstacle information; The efficiency cost data of the first candidate reference line is determined based on the speed information; The comfort cost data of the first candidate reference line is determined based on the acceleration information; The total cost data is determined based on the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data.

[0011] Optionally, determining the total cost data based on the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data includes: Determine the weight data corresponding to different types of costs; The total cost data is determined by weighting and summing the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data according to the weighted data.

[0012] Optionally, the determination of weight data corresponding to different types of costs includes: Obtain scene data of the scene where the autonomous vehicle is located; Determine the scene complexity data based on the scene data; Based on the basic weights and the scenario complexity data, determine the weight data corresponding to different types of costs in different scenarios.

[0013] Optionally, smoothing the second candidate reference line to obtain the target reference line for the autonomous vehicle includes: The second candidate reference line is further smoothed according to the preset B-spline curve to obtain the target reference line of the autonomous vehicle.

[0014] Optionally, the device includes: The data acquisition module is used to acquire navigation information and SD maps for autonomous vehicles; A spatial feature data extraction module is used to extract spatial feature data based on the SD map; A time-series feature data extraction module is used to obtain time-series feature data based on the navigation information; The feature data fusion determination module is used to fuse the spatial feature data and the temporal feature data to obtain fused feature data; The target reference line generation module is used to generate the target reference line of the autonomous vehicle based on the fused feature data.

[0015] An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the method for generating autonomous driving reference lines as described in any one of claims 1 to 8.

[0016] A computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform the method for generating autonomous driving reference lines as described above.

[0017] A vehicle that includes the electronic equipment described above.

[0018] The beneficial effects of the present invention are as follows: The present invention can combine SD map and navigation information to extract time and space features, and then perform feature fusion, thereby generating a target reference line for autonomous vehicles based on the fused feature data, and generating a reasonable and accurate reference line. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the steps of a method for generating autonomous driving reference lines provided in an embodiment of the present invention. Figure 2 This is a flowchart of another method for generating autonomous driving reference lines provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an automatic driving reference line generation device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0021] It should be noted that the embodiments of the present invention may involve the use of user data. In practical applications, user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, provided that it complies with the applicable laws and regulations of the country (e.g., with the user's explicit consent, with the user being properly notified, etc.).

[0022] Reference Figure 1 The diagram illustrates a flowchart of a method for generating autonomous driving reference lines according to an embodiment of the present invention, specifically including the following steps: Step S101: Obtain navigation information and SD map of the autonomous vehicle; In this embodiment of the invention, autonomous vehicles require accurate reference lines for vehicle control during operation to ensure driving safety.

[0023] Vehicles store SD (Standard Definition) maps locally or in the cloud. SD maps are lightweight navigation maps designed for mass-produced intelligent driving, with semantics and topology at their core. Through an industrialized model of crowdsourced data collection, cloud fusion, and differential updates, they achieve core advantages of low cost, high coverage, and high freshness. SD maps possess the core characteristics of being lightweight, topological, and semantically semantic.

[0024] Autonomous vehicles can set a destination while driving, thus generating navigation information. In practical applications, the navigation information can change dynamically during the vehicle's journey.

[0025] In the embodiments of the present invention, the navigation information reflects the actual route that the vehicle needs to travel. During actual driving, it is necessary to determine the actual end-to-end reference line based on the navigation information.

[0026] In one embodiment of the present invention, after obtaining the SD map and navigation information, the SD map and navigation information can be preprocessed. The purpose of preprocessing is to facilitate subsequent calculations using the SD map and navigation information. Preprocessing may include unified coordinate system transformation.

[0027] First, autonomous vehicles can standardize the input navigation path information. Let the navigation path be P_nav = {(x_i, y_i, θ_i)}, where (x_i, y_i) represent the coordinates of the path points and θ_i represents the heading angle. The path is resampled with equal arc lengths, and the sampling interval is set to d_s = 2.0 meters.

[0028] The SD map information includes lane centerlines, lane boundaries, and semantic labels. The lane centerline is represented as L_lane ={(x_j, y_j, w_j, s_j)}, where w_j is the lane width and s_j is the semantic label. Local extraction is performed on the SD map, focusing on map elements within a 200-meter radius in front of and 50-meter behind the vehicle's current position. These map elements can include six categories: lane lines, centerlines, curbs, sidewalks, stop lines, and ground arrows.

[0029] Establish a unified coordinate system, the Frenet coordinate system, with the vehicle's current position as the origin and the vehicle's heading as the positive x-axis. The coordinate transformation formula is: s = ∫0 t √((dx / dt)² + (dy / dt)²) dt d = (x - x_ref(s)) × sin(θ_ref(s)) - (y - y_ref(s)) × cos(θ_ref(s)) Where s is the vertical coordinate, d is the horizontal coordinate, and (x_ref(s), y_ref(s), θ_ref(s)) are the position and heading of the corresponding point on the reference line.

[0030] The Frenet coordinate system is a coordinate system that describes the position of an object near a reference line. It provides a more intuitive description of vehicle driving behavior than the global Cartesian coordinate system (X, Y). The vertical coordinate is the displacement along the reference line, usually denoted as s; the horizontal coordinate is the displacement perpendicular to the reference line, usually denoted as d. The given formula transforms the vehicle's position in the global Cartesian coordinate system (X, Y) to the Frenet coordinate system (s, d) based on the reference line.

[0031] Step S102: Extract spatial feature data based on the SD map; In this embodiment of the invention, a first feature extraction model for extracting spatial feature information can be set in the vehicle. After obtaining the SD map, the vehicle can use the trained first feature extraction model to extract spatial technical features from the obtained SD map.

[0032] Specifically, a convolutional neural network (i.e., the first feature extraction model) can be used to extract spatial features from the SD map. The SD map is rasterized into a 200×200 pixel bird's-eye view, with each pixel representing a 0.2×0.2 meter area. Features are extracted using the ResNet-18 architecture, outputting a 512-dimensional feature vector F_map.

[0033] Among them, the spatial feature F_map of the SD map is a 512-dimensional feature vector extracted by a CNN network, which specifically contains the following spatial semantic information: Geometric features (128 dimensions): lane curvature, width, and direction information; curb position and height information.

[0034] Topological features (128 dimensions): lane connection relationships, intersection structure, lane merging and separation information.

[0035] Semantic features (128 dimensions): stop line position, ground arrow direction, pedestrian area markings.

[0036] Spatial relationship characteristics (128 dimensions): the relative position, distance, and angle relationships between various map elements.

[0037] Step S103: Obtain temporal feature data based on the navigation information; After obtaining the navigation information, a recurrent neural network (RNN) can be used to extract temporal features. The navigation path point sequence is input into an LSTM network with a hidden layer dimension of 256, outputting a 128-dimensional feature vector F_nav. The RNN can be a pre-trained network model.

[0038] Vehicle state information can include position, velocity, acceleration, and heading angle, forming a 9-dimensional state vector v_ego = [x, y, v_x, v_y, a_x, a_y, θ, ω, κ], where κ is the curvature.

[0039] Step S104: The spatial feature data and the temporal feature data are fused to obtain fused feature data; After extracting spatial and temporal features, the two types of features can be fused to obtain fused feature data.

[0040] In practical applications, an attention mechanism can be introduced to fuse spatial feature data and temporal feature data to obtain fused feature data. The fusion process based on the attention mechanism is as follows: Design a multi-head attention mechanism to fuse information from different modalities. Define the query matrix Q, the key matrix K, and the value matrix V: Q = F_ego W_Q, K = [F_map; F_nav] W_K, V = [F_map; F_nav] W_V Where W_Q, W_K, and W_V are learnable weight matrices, and F_ego is the vehicle state feature.

[0041] The query matrix Q represents "what information I want".

[0042] Q = F_ego W_Q, where F_ego is a 9-dimensional vehicle state feature, and W_Q is a 9×512 weight matrix that maps the vehicle state to a query vector. This formula indicates which map and navigation information needs to be considered under the current vehicle state.

[0043] Key matrix K: Represents "what information I have".

[0044] K = [F_map; F_nav] W_K, concatenates map features and navigation features, where [F_map; F_nav] is a 640-dimensional vector (512+128). W_K is a 640×512 weight matrix. This formula can represent the index of available map and navigation information.

[0045] Value matrix V: Represents the "specific content of the information".

[0046] V = [F_map; F_nav] W_V, with the same structure as K. W_V is a 640×512 weight matrix, which can represent the actual information content to be extracted.

[0047] The formula for calculating attention weights is: Attention(Q,K,V) = softmax(QK^T / √d_k)V Where d_k is the dimension of the key vector. Eight attention heads are used, each with a dimension of 64.

[0048] QK^T: Calculate the similarity between the query and the key to obtain a 512×512 attention score matrix.

[0049] √d_k: scaling factor, d_k=512, to avoid the gradient of the softmax function being too small.

[0050] softmax: Converts the scores into a probability distribution, ensuring that the weights sum to 1.

[0051] Final result: Weighted feature vector, dimension 512.

[0052] The specific data processing procedure is illustrated below: Assume the current scenario: A vehicle is preparing to turn left at an intersection. Input data: Vehicle status: v_ego = [100, 50, 15, 0, 0, -0.5, π / 4, 0.1, 0.05] Map features: F_map contains intersection geometry information and left-turn lane information. Navigation characteristics: F_nav contains the path instruction "turn left ahead". Feature processing: Q = v_ego × W_Q → 512-dimensional query vector (Pay attention to information related to left turn) K = [F_map; F_nav] × W_K → 512-dimensional bond vector V = [F_map; F_nav] × W_V → 512-dimensional value vector Attention calculation: Attention weights = softmax([0.8, 0.2]) # Assuming a weight of 0.8 for map information and 0.2 for navigation information. Fusion features = 0.8 × V_map + 0.2 × V_nav Multimodal information fusion mechanism: Eight attention heads are used for parallel processing, each focusing on a different dimension of information.

[0053] First 1-2: Focus on geometric constraints (lane lines, curvature).

[0054] First 3-4: Focus on semantic constraints (stop lines, markers).

[0055] First 5-6: Focus on navigation instructions (turn, target).

[0056] First 7-8: Focus on safety constraints (obstacles, boundaries).

[0057] Finally, the fusion formula: F_fused = Concat(head1, head2, ..., head8) × W_O Where W_O is a 512×512 output projection matrix.

[0058] Step S105: Generate the target reference line of the autonomous vehicle based on the fused feature data.

[0059] After obtaining the fused feature data, which combines navigation information and SD map information, a reasonable target reference line can be generated for autonomous vehicles.

[0060] In this embodiment of the invention, navigation information and SD map of autonomous vehicle can be obtained; spatial feature data can be extracted based on the SD map; temporal feature data can be obtained based on the navigation information; the spatial feature data and the temporal feature data can be fused to obtain fused feature data; and the target reference line of autonomous vehicle can be generated according to the fused feature data. Thus, end-to-end reference line generation can be achieved by combining navigation information and SD map to realize reasonable and accurate planning reference line.

[0061] Reference Figure 2 The diagram illustrates a flowchart of another method for generating autonomous driving reference lines according to an embodiment of the present invention, which specifically includes the following steps: Step S201: Obtain navigation information and SD map of the autonomous vehicle; Step S202: Extract spatial feature data based on the SD map; Step S203: Obtain temporal feature data based on the navigation information; Step S204: The spatial feature data and the temporal feature data are fused to obtain fused feature data; Step S205: Generate a first candidate reference line based on the fusion features; After obtaining the fusion features, candidate points can be determined based on the fusion features, and then the first candidate reference line can be generated based on the candidate points.

[0062] In one embodiment of the present invention, generating a first candidate reference line based on the fusion feature includes: inputting the fusion feature into a preset multilayer perceptron and outputting a plurality of candidate points corresponding to the fusion feature; the multilayer perceptron being used to output the corresponding candidate points based on the input fusion feature; and generating a first candidate reference line according to the candidate points.

[0063] In practical applications, the fused feature input to a pre-defined multilayer perceptron mechanism can be mapped to candidate point coordinates through a fully connected layer: [x_cand, y_cand, θ_cand] = MLP(F_fused) MLP stands for Multilayer Perceptron, which contains three hidden layers, each with 256 neurons, and uses ReLU as the activation function.

[0064] After obtaining the fused features, the fused features F_fused can be input into the MLP network. The processing in the MLP is: 512 → 256 → 256 → 256 → 3 (outputting x, y, θ coordinates).

[0065] Generate N=20 candidate points, predict in the time domain of T=5 seconds, and use a time step of Δt=0.25 seconds. The candidate point sequence is R_cand = {(x_i, y_i, θ_i, κ_i, v_i)}, where i=1,2,...,N.

[0066] The above-mentioned multi-layer perceptron design ensures that map information, navigation information, and vehicle status are effectively integrated and utilized during the reference line generation process.

[0067] Step S206: Optimize the first candidate reference line according to the preset optimization objective function to obtain the second candidate reference line; In practical applications, optimization can be performed on the initial generation of the first candidate reference line. Specifically, an optimization function can be set to perform multi-objective optimization.

[0068] In one embodiment of the present invention, optimizing the first candidate reference line according to a preset optimization objective function to obtain a second candidate reference line includes the following sub-steps: Sub-step 11: Create the kinematic model and reference line constraints of the autonomous vehicle; In practical applications, a vehicle kinematic model can be established: = v cos(θ), = v sin(θ), = v κ Constraints can include maximum curvature constraints, maximum rate of change of curvature constraints, and maximum lateral acceleration constraints. By setting constraints, it can be ensured that the generated reference line is a reasonable reference line.

[0069] One constraint in this embodiment of the invention can be set as follows: maximum curvature constraint |κ| ≤ κ_max = 0.2rad / m, maximum rate of change of curvature constraint | | ≤ _max = 0.1 rad / (m·s), maximum lateral acceleration constraint |a_lat| = v²κ ≤ a_lat_max = 3.0 m / s².

[0070] Sub-step 12: Determine the total cost data corresponding to the preset optimization objective function based on the kinematic model and the constraints. The optimization objective function is a multi-objective cost function, which can calculate the total cost of the optimized reference line based on multiple optimization objectives, and finally select the reference line with the minimum cost as the first candidate reference line.

[0071] Sub-step 13: Determine a second candidate reference line from the first candidate reference line based on the total cost data.

[0072] In one embodiment of the present invention, determining the total cost data corresponding to the preset optimization objective function based on the kinematic model and the constraints includes: determining curvature information, obstacle information, velocity information, and acceleration information of the first candidate reference line based on the kinematic model and the constraints; determining smoothness cost data of the first candidate reference line according to the curvature information; determining safety cost data of the first candidate reference line according to the obstacle information; determining efficiency cost data of the first candidate reference line according to the velocity information; determining comfort cost data of the first candidate reference line according to the acceleration information; and determining the total cost data based on the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data.

[0073] In practical applications, the optimization objective function can be composed of smoothness cost data, safety cost information, efficiency cost data, and comfort cost data.

[0074] For example, the optimization objective function J can be predefined: J = w1J_smooth + w2J_safety + w3J_efficiency + w4J_comfort in: J_smooth = ∫( ²)ds, smoothness cost.

[0075] in, ² can represent the curvature of a trajectory (or a higher-order derivative, such as the rate of change of curvature). Typically, a smoothness cost penalizes the curvature or rate of change of curvature of a trajectory, making the trajectory smoother. In autonomous driving, a smooth trajectory means smoother steering wheel turns and a more comfortable ride.

[0076] J_safety = ∑exp(-d_obs / σ), where d_obs is the distance to the obstacle.

[0077] For each obstacle (or each discrete point), calculate its distance d_obs to the obstacle, then penalize trajectories that are too close to the obstacle exponentially. σ is a parameter used to adjust the sensitivity of safety. When the distance d_obs is small, the exponential term will be large, thus strongly penalizing such trajectories. The summation symbol may represent summing over multiple obstacles or multiple points on the trajectory.

[0078] J_efficiency = ∫(v_ref - v)²dt, where J is the efficiency cost.

[0079] This cost function encourages vehicles to travel at a reference speed v_ref. A cost is incurred when the actual speed v deviates from the reference speed. The integral represents the accumulated speed deviation over a period of time.

[0080] J_comfort = ∫(a²_lat + a²_lon)dt, the cost of comfort.

[0081] Here, a_lat and a_long represent lateral and longitudinal acceleration, respectively. Comfort comes at the cost of larger accelerations (both lateral and longitudinal), as rapid acceleration, deceleration, or sharp turns can all lead to discomfort. Minimizing the sum of the squares of accelerations can improve passenger comfort.

[0082] In one embodiment of the present invention, determining the total cost data based on the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data includes: determining weight data corresponding to different types of costs; and performing a weighted summation of the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data based on the weight data to determine the total cost data.

[0083] In practical applications, after calculating various cost data, a weighted sum can be performed by combining these cost data. The weight data can be set according to the type of cost data; for example, the weight coefficients can be w1=1.0, w2=5.0, w3=0.5, and w4=2.0. Alternatively, the weight data can be dynamically set according to the complexity of the actual scenario, meaning that the weights corresponding to the same type of cost change dynamically under different scenario complexities.

[0084] In one embodiment of the present invention, the weight data can be dynamically set. Specifically, the autonomous vehicle can acquire scene data of the scene in which the autonomous vehicle is located; determine scene complexity data based on the scene data; and determine weight data corresponding to different types of costs under different scenarios based on the basic weights and the scene complexity data.

[0085] In practical applications, scenario complexity metrics can be predefined: C_scene = α1N_vehicle + α2N_pedestrian + α3C_curve + α4V_traffic Where N_vehicle is the number of vehicles in the surrounding area, N_pedestrian is the number of pedestrians, C_curve is the road curvature, and V_traffic is the traffic flow speed. The coefficients α1=0.3, α2=0.5, α3=0.2, and α4=0.1.

[0086] Weight adaptive adjustment formula: w2= w2_base × (1 + 0.5 × C_scene) w4= w4_base × (1 + 0.2 × C_scene) Among them, w2_base=5.0 and w4_base=2.0 are the base weights.

[0087] Step S207: Smooth the second candidate reference line to obtain the target reference line of the autonomous vehicle.

[0088] After obtaining the second candidate reference line, it can be smoothed to simulate a more reasonable and accurate reference line. Specifically, the second candidate reference line can be smoothed according to a preset B-spline curve to obtain the target reference line of the autonomous vehicle.

[0089] First, we can define the control points as P = {P0, P1, ..., P}. n}; The expression for a cubic B-spline curve is defined as: C(t) = ∑ i=0 nN i ,3(t)P i N i ,3(t) is a cubic B-spline basis function: N i ,3(t) = (tt i ) / (t i+3 -t i ) × N i ,2(t) + (t i+4 -t) / (t i+4 -t i+1 ) × N i+1 ,2(t) In this embodiment of the invention, the node vectors can be set to a uniform distribution, and the control points are determined by fitting the optimized reference point sequence using the least squares method.

[0090] The final output reference line includes position information (x, y), heading angle θ, curvature κ, velocity suggestion v_ref, and confidence score conf. The reference line sampling density can be set according to actual needs; for example, the sampling density can be set to one point every 0.5 meters, with a total length of 150 meters.

[0091] In this embodiment of the invention, features can be extracted and fused from SD maps and navigation information, and then the target reference line can be obtained after optimization and smoothing. This allows for reasonable end-to-end reference line planning.

[0092] Reference Figure 3 The diagram illustrates a flowchart of another autonomous driving reference line generation device provided in an embodiment of the present invention, which specifically includes the following modules: The data acquisition module 301 is used to acquire navigation information and SD maps of autonomous vehicles; Spatial feature data extraction module 302 is used to extract spatial feature data based on the SD map; The temporal feature data extraction module 303 is used to obtain temporal feature data based on the navigation information; The feature data fusion determination module 304 is used to fuse the spatial feature data and the temporal feature data to obtain fused feature data. The target reference line generation module 305 is used to generate the target reference line of the autonomous vehicle based on the fused feature data.

[0093] In one embodiment of the present invention, the target reference line generation module 305 may include: The first candidate reference line generation submodule is used to generate a first candidate reference line based on the fusion features; The second candidate reference line generation submodule is used to optimize the first candidate reference line according to a preset optimization objective function to obtain the second candidate reference line. The target reference line generation submodule is used to smooth the second candidate reference line to obtain the target reference line of the autonomous vehicle.

[0094] In one embodiment of the present invention, the first candidate reference line generation submodule may include: A candidate point generation unit is used to input the fused features into a preset multilayer perceptron and output multiple candidate points corresponding to the fused features; the multilayer perceptron is used to output corresponding candidate points based on the input fused features. The first candidate reference line generation unit is used to generate a first candidate reference line according to the candidate points.

[0095] In one embodiment of the present invention, the second candidate reference line generation submodule may include: A constraint creation unit is used to create constraints for the kinematic model and reference lines of the autonomous vehicle. The total cost data determination unit is used to determine the total cost data corresponding to the preset optimization objective function based on the kinematic model and the constraints. The second candidate reference line generation unit is used to determine a second candidate reference line from the first candidate reference lines based on the total cost data.

[0096] In one embodiment of the present invention, the total cost data determination unit may include: The reference line data determination subunit is used to determine the curvature information, obstacle information, velocity information, and acceleration information of the first candidate reference line based on the kinematic model and the constraint conditions. A smoothing cost determination subunit is used to determine the smoothing cost data of the first candidate reference line according to the curvature information; A safety cost determination subunit is used to determine the safety cost data of the first candidate reference line based on the obstacle information. An efficiency cost determination subunit is used to determine the efficiency cost data of the first candidate reference line based on the speed information. A comfort cost determination subunit is used to determine the comfort cost data of the first candidate reference line according to the acceleration information; The total cost determination subunit is used to determine the total cost data based on the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data.

[0097] In one embodiment of the present invention, the total cost determination subunit may include: The weight data determination block is used to determine the weight data corresponding to different types of costs; The total cost data determination block is used to determine the total cost data by weighted summation of the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data according to the weight data.

[0098] In one embodiment of the present invention, the weight data determination block may include: The scene data acquisition sub-block is used to acquire scene data of the scene in which the autonomous vehicle is located; The scene complexity determination sub-block is used to determine scene complexity data based on the scene data; The weight data determination sub-block is used to determine the weight data corresponding to different types of costs in different scenarios based on the basic weights and the scenario complexity data.

[0099] In one embodiment of the present invention, the target reference line generation submodule may include: The target reference line determination unit is used to smooth the second candidate reference line according to a preset B-spline curve to obtain the target reference line of the autonomous vehicle.

[0100] In this embodiment of the invention, navigation information and SD map of autonomous vehicle can be obtained; spatial feature data can be extracted based on the SD map; temporal feature data can be obtained based on the navigation information; the spatial feature data and the temporal feature data can be fused to obtain fused feature data; and the target reference line of autonomous vehicle can be generated according to the fused feature data. Thus, end-to-end reference line generation can be achieved by combining navigation information and SD map to realize reasonable and accurate planning reference line.

[0101] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment. This invention also provides an electronic device, such as... Figure 4 As shown, it includes a processor 401, a device interface 402, a memory 403, and a bus 404; Memory 403 is used to store computer programs; The processor 401 performs the above steps when executing the program stored in the memory 403.

[0102] The bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0103] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0104] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0105] The present invention also provides a storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the vehicle parking method of the foregoing embodiments.

[0106] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0107] The algorithms and displays provided herein are not inherently related to any particular computer, virtual device, or other equipment. The structure required to construct such a device is readily apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0108] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0109] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0110] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0111] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention can also be implemented as a device or apparatus program for performing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0112] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0113] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0115] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0116] It should be noted that the various data-related processes in the embodiments of this application are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

Claims

1. A method for generating reference lines for autonomous driving, characterized in that, The method includes: Obtain navigation information and SD maps from autonomous vehicles; Spatial feature data is extracted based on the SD map; Temporal feature data is obtained based on the navigation information; The spatial feature data and the temporal feature data are fused to obtain fused feature data; The fused features are input into a preset multilayer perceptron, which outputs multiple candidate points corresponding to the fused features; the multilayer perceptron is used to output corresponding candidate points based on the input fused features. A first candidate reference line is generated based on the candidate points; Create the kinematic model and reference line constraints for the autonomous vehicle; Based on the kinematic model and the constraints, determine the total cost data corresponding to the preset optimization objective function; A second candidate reference line is determined from the first candidate reference line based on the total cost data; The second candidate reference line is smoothed to obtain the target reference line of the autonomous vehicle.

2. The method according to claim 1, characterized in that, The determination of the total cost data corresponding to the preset optimization objective function based on the kinematic model and the constraints includes: Based on the kinematic model and the constraints, the curvature information, obstacle information, velocity information, and acceleration information of the first candidate reference line are determined. The smoothness cost data of the first candidate reference line is determined based on the curvature information; The safety cost data of the first candidate reference line is determined based on the obstacle information; The efficiency cost data of the first candidate reference line is determined based on the speed information; The comfort cost data of the first candidate reference line is determined based on the acceleration information; The total cost data is determined based on the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data.

3. The method according to claim 2, characterized in that, The step of determining the total cost data based on the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data includes: Determine the weight data corresponding to different types of costs; The total cost data is determined by weighting and summing the smoothness cost data, the safety cost information, the efficiency cost data, and the comfort cost data according to the weighted data.

4. The method according to claim 3, characterized in that, The weight data for determining different types of costs includes: Obtain scene data of the scene where the autonomous vehicle is located; Determine the scene complexity data based on the scene data; Based on the basic weights and the scenario complexity data, determine the weight data corresponding to different types of costs in different scenarios.

5. The method according to claim 1, characterized in that, The step of smoothing the second candidate reference line to obtain the target reference line for the autonomous vehicle includes: The second candidate reference line is smoothed according to a preset B-spline curve to obtain the target reference line of the autonomous vehicle.

6. An apparatus for generating reference lines for autonomous driving, characterized in that, The device includes: The data acquisition module is used to acquire navigation information and SD maps for autonomous vehicles; A spatial feature data extraction module is used to extract spatial feature data based on the SD map; A time-series feature data extraction module is used to obtain time-series feature data based on the navigation information; The feature data fusion determination module is used to fuse the spatial feature data and the temporal feature data to obtain fused feature data; A target reference line generation module is used to generate a target reference line for the autonomous vehicle based on the fused feature data. The target reference line generation module includes: The first candidate reference line generation submodule is used to generate a first candidate reference line based on the fusion features; The second candidate reference line generation submodule is used to optimize the first candidate reference line according to a preset optimization objective function to obtain the second candidate reference line. The target reference line generation submodule is used to smooth the second candidate reference line to obtain the target reference line of the autonomous vehicle. The first candidate reference line generation submodule includes: A candidate point generation unit is used to input the fused features into a preset multilayer perceptron and output multiple candidate points corresponding to the fused features; the multilayer perceptron is used to output corresponding candidate points based on the input fused features. The first candidate reference line generation unit is used to generate a first candidate reference line according to the candidate points; The second candidate reference line generation submodule includes: A constraint creation unit is used to create constraints for the kinematic model and reference lines of the autonomous vehicle. The total cost data determination unit is used to determine the total cost data corresponding to the preset optimization objective function based on the kinematic model and the constraints. The second candidate reference line generation unit is used to determine a second candidate reference line from the first candidate reference lines based on the total cost data.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the method for generating autonomous driving reference lines as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal is able to perform the method for generating autonomous driving reference lines as described in any one of claims 1 to 5.

9. A vehicle, characterized in that, The vehicle includes the electronic equipment as described in claim 7.

Images