Smooth processing method and device for automatic driving path, equipment and medium

Abstract

The invention discloses a smoothing processing method and device for an automatic driving path, equipment and a medium. The method belongs to the technical field of automatic driving path planning, and comprises the following steps: obtaining at least one planning path output by an end-to-end model; wherein the planned path is generated by an end-to-end automatic driving model based on a streaming image sequence and environment feature information acquired by vehicle-mounted equipment; classifying a current driving scene based on the environment feature information and a context sensing decision module to determine a scene label; and calling a preset multi-modal Kalman filter library to perform smoothing processing on the at least one planned path based on the scene label to obtain a target planned path. The technical effect of improving the safety of the automatic driving path and the riding comfort is achieved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving path planning technology, and in particular to the smoothing of autonomous driving paths. Background Technology

[0002] The end-to-end autonomous driving model establishes a mapping from perception to planned trajectory through a large neural network model. It uses raw sensor information as input signals and directly outputs future planned trajectory information.

[0003] In existing autonomous driving trajectory planning, sensor information is typically used directly as model input. Through a network paradigm of imitation learning, an end-to-end autonomous driving model outputs the planned path. Simultaneously, vectorized maps and bird's-eye view targets are embedded as perception aids into the end-to-end model to enhance its perception of the surrounding environment. Alternatively, DSRC technology can be used to automatically identify moving vehicles and acquire their speed, position, and time information, enabling information exchange between the onboard system and roadside units. Next, based on the collected information, quantization formulas are used to quantize the azimuth angles of the roadside units and the onboard system, thus obtaining the current lane information of the vehicle. Finally, autoregressive moving average and adaptive Kalman filtering are combined to predict acceleration and position, assisting in vehicle trajectory tracking.

[0004] End-to-end models rely on data-driven approaches to directly output trajectories without considering the actual motion characteristics of the vehicle. The planned trajectories suffer from insufficient smoothness and do not fully meet vehicle dynamics constraints. While adaptive Kalman filters can adjust parameters based on noise characteristics, their trajectory smoothing performance is poor in certain scenarios. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for smoothing autonomous driving paths, so as to achieve target planning path generation that matches the driving scenario.

[0006] According to one aspect of the present invention, a method for smoothing autonomous driving paths is provided, the method comprising:

[0007] Obtain at least one planned path output by the end-to-end model; wherein the planned path is generated by the end-to-end autonomous driving model based on the streaming image sequence and environmental feature information collected by the on-board equipment;

[0008] Based on the environmental feature information and the context-aware decision module, the current driving scenario is classified to determine the scenario label;

[0009] The preset multimodal Kalman filter library is invoked to smooth at least one planned path based on the scene label in order to obtain the target planned path.

[0010] According to another aspect of the present invention, a smoothing processing apparatus for an autonomous driving path is provided, the apparatus comprising:

[0011] The output path acquisition module is used to acquire at least one planned path output by the end-to-end model; wherein, the planned path is generated by the end-to-end autonomous driving model based on the streaming image sequence and environmental feature information collected by the on-board equipment;

[0012] The scene label determination module is used to classify the current driving scene based on the environmental feature information and the context-aware decision module in order to determine the scene label;

[0013] The smoothing module is used to call a preset multimodal Kalman filter library to smooth at least one planned path based on the scene label in order to obtain the target planned path.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] At least one processor;

[0016] and memory that is communicatively connected to at least one processor;

[0017] The memory stores a computer program that can be executed by at least one processor, which enables the at least one processor to perform the smoothing method for autonomous driving paths according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the smoothing method for an autonomous driving path according to any embodiment of the present invention.

[0019] The technical solution of this invention involves obtaining at least one planned path output by an end-to-end model. This planned path is generated by the end-to-end autonomous driving model based on streaming image sequences and environmental feature information collected by onboard equipment. The current driving scenario is classified based on the environmental feature information and a context-aware decision module to determine scenario labels. A preset multimodal Kalman filter library is then invoked to smooth the at least one planned path based on the scenario labels to obtain the target planned path. This solves the technical problems of insufficient smoothness and poor scenario adaptability of the end-to-end model output path in complex scenarios, achieving the technical effect of improving the safety and ride comfort of autonomous driving paths.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a method for smoothing autonomous driving paths provided in an embodiment of the present invention;

[0023] Figure 2a A flowchart of another method for smoothing autonomous driving paths provided in an embodiment of the present invention;

[0024] Figure 2b A flowchart of the model path planning process for another method of smoothing autonomous driving paths provided in an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of the structure of an autonomous driving path smoothing device provided in an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of the structure of an electronic device for implementing a method for smoothing autonomous driving paths according to an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Figure 1 This is a flowchart illustrating a method for smoothing an autonomous driving path according to an embodiment of the present invention. This embodiment is applicable to situations where the autonomous driving path is determined. The method can be executed by an autonomous driving path device, which can be implemented in hardware and / or software and can be configured in an electronic device. For example... Figure 1 As shown, the method specifically includes the following steps:

[0030] S110. Obtain at least one planned path output by the end-to-end model; wherein the planned path is generated by the end-to-end autonomous driving model based on the streaming image sequence and environmental feature information collected by the on-board equipment.

[0031] In this context, the end-to-end model can be understood as a one-stop model directly from input to output. The planned path can be understood as the route the vehicle will initially plan to travel, as envisioned by the autonomous driving model. Onboard equipment can be understood as sensors, cameras, and other devices installed on the vehicle. Streaming image sequences can be understood as image data captured continuously by onboard cameras and arranged in chronological order. Environmental feature information can be understood as various information describing the vehicle's surrounding environment, such as traffic conditions and weather.

[0032] Specifically, preliminary planned routes are obtained from the end-to-end model. These routes are candidate routes for vehicle driving directly generated by the model after analyzing real-time images and environmental data collected by the on-board equipment.

[0033] In some possible implementations, the environmental feature information includes at least traffic participant density, road curvature, weather conditions, and intersection traffic signals.

[0034] Among these, traffic participant density can be understood as the number of pedestrians, vehicles, and other traffic participants within a unit area. Road curvature can be understood as the degree of road curvature. Weather conditions can be understood as the current weather conditions, such as sunny, rainy, or foggy. Intersection traffic signals can be understood as the status of traffic indicator signals such as traffic lights at intersections.

[0035] S120. Based on the environmental feature information and the context-aware decision module, the current driving scenario is classified to determine the scenario label.

[0036] The context-aware decision-making module can be understood as a module that can determine the current scenario by combining environmental information. The classification of driving scenarios can be understood as dividing the driving environment into different types according to certain rules, such as urban road scenarios, highway scenarios, etc. Scenario tags can be understood as identifiers used to identify the type of the current driving scenario, such as urban road congestion on a sunny day.

[0037] Specifically, the context-aware decision-making module analyzes the environmental features around the vehicle, such as traffic conditions, road types, and weather, and classifies the current driving environment into a specific type according to preset classification rules, ultimately generating a label that can accurately identify the scene.

[0038] Preferably, historical scene data and real-time data can be combined during classification to reduce classification errors caused by sudden environmental changes.

[0039] In some possible implementations, the classification of the current driving scenario based on the environmental feature information and the context-aware decision module to determine the scenario label includes:

[0040] The context-aware decision-making module extracts features from the environmental feature information to obtain traffic flow density features, road structure features, weather state features, and traffic signal features.

[0041] Based on preset scene classification rules, the traffic flow density features, road structure features, and weather state features are classified respectively to obtain traffic level classification results, road type classification results, and weather state classification results.

[0042] The combined result of the traffic level classification, the road type classification, and the weather condition classification is used as the classification of the current driving scenario, and a scenario label containing the combined result is generated.

[0043] Traffic flow density characteristics can be understood as features that reflect the degree of traffic congestion.

[0044] Road structure features can be understood as characteristic information describing the road's own structure, such as road type, number of lanes, and speed limit. Traffic signal features can be understood as characteristic information reflecting the traffic signal status and cycle of intersections. Preset scenario classification rules can be understood as explicit rules set in advance based on experience to classify traffic levels, road types, and weather conditions; this embodiment does not impose specific restrictions on them. Traffic level classification results can be understood as classification results of traffic flow congestion, such as three levels: congested, moderate, and smooth. Road type classification results can be understood as classification results of road types, such as three types: highway, urban road, and rural road. Weather condition classification results can be understood as classification results of weather conditions, such as four states: sunny, rainy, foggy, and snowy. Combined results can be understood as a combination of traffic level, road type, and weather condition classification results.

[0045] Specifically, the context-aware decision-making module first extracts targeted features from environmental characteristics. From the raw data, it extracts traffic flow density features reflecting traffic congestion, such as the number and distribution of vehicles per unit road segment; road structure features reflecting road attributes, such as the number of lanes, speed limit signs, and road materials; weather condition features representing weather conditions, such as precipitation intensity and visibility; and traffic signal features reflecting intersection traffic rules, such as traffic light cycles and phase changes. Next, based on preset scenario classification rules, such as >50 vehicles per kilometer indicating congestion, four or more lanes in both directions indicating arterial roads, and precipitation >5 mm / h indicating rainy days, the extracted traffic flow density features, road structure features, and weather condition features are quantified and classified to obtain specific traffic level, road type, and weather condition classification results. Finally, these three classification results are combined to fully define the current driving scenario and generate a scenario label containing this combined information.

[0046] Preferably, the classification dimensions cover at least road type, weather conditions, and traffic flow density.

[0047] For example, the traffic flow density characteristics are divided into three levels: congested, normal, and smooth; the road structure characteristics are divided into three types: highway, urban road, and rural road; and the weather condition characteristics are divided into four states: sunny, rainy, foggy, and snowy.

[0048] In some possible implementations, the context-aware decision module uses a lightweight decision tree or a deep learning classification model to classify the scene.

[0049] Lightweight decision trees can be understood as decision tree models with simple structure, low computational cost, and fast running speed. Deep learning classification models can be understood as classification models based on deep learning algorithms, capable of handling complex environmental feature data, with high classification accuracy, and suitable for scenarios requiring high precision in scene judgment.

[0050] S130. Call the preset multimodal Kalman filter library to smooth at least one planned path based on the scene label to obtain the target planned path.

[0051] The pre-defined multimodal Kalman filter library can be understood as a collection of Kalman filters with different parameters, prepared in advance to handle paths in different scenarios. The target planning path can be understood as a smoothed path.

[0052] Specifically, appropriate filters are matched from the filter library based on scene labels to smooth and correct the initially planned path, making the path more in line with vehicle dynamics and scene requirements.

[0053] Preferably, the path effect can be monitored in real time during the processing, and the filter combination method can be dynamically adjusted.

[0054] In some possible implementations, the optimized parameters of each filter in the multimodal Kalman filter library include the state transition matrix, the process noise covariance matrix, and the observation noise covariance matrix, and the parameters of different filters are optimized for at least one scenario combination.

[0055] The state transition matrix can be understood as a matrix describing the change of system state over time; the process noise covariance matrix can be understood as a matrix describing the statistical characteristics of various noises during system operation; the observation noise covariance matrix can be understood as a matrix describing the statistical characteristics of noise in the observation data; and the scene combination can be understood as a combination of different traffic levels, road types, and weather conditions.

[0056] The technical solution of this invention involves obtaining at least one planned path output by an end-to-end model. This planned path is generated by the end-to-end autonomous driving model based on streaming image sequences and environmental feature information collected by onboard equipment. The current driving scenario is classified based on the environmental feature information and a context-aware decision module to determine scenario labels. A preset multimodal Kalman filter library is then invoked to smooth the at least one planned path based on the scenario labels to obtain the target planned path. This solves the technical problems of insufficient smoothness and poor scenario adaptability of the end-to-end model output path in complex scenarios, achieving the technical effect of improving the safety and ride comfort of autonomous driving paths.

[0057] Figure 2aThis is a flowchart of another application installation control method for an in-vehicle system provided by an embodiment of the present invention. Based on the above embodiments, this embodiment focuses on how to call a preset multimodal Kalman filter library to smooth at least one planned path based on the scene labels, thereby refining the target planned path. Specific implementation details can be found in the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here. Figure 2a As shown, the method specifically includes the following steps:

[0058] S1301. Call the preset multimodal Kalman filter library and at least one planning path to calculate the matching degree between the scene label and each filter and determine the corresponding confidence level.

[0059] The matching degree can be understood as a measure of how well the scene label fits the filter. The confidence degree can be understood as a measure of the reliability of each filter in processing the current scene, based on the matching degree.

[0060] Specifically, a specific algorithm is used to evaluate the fit between each filter and the current scene, and then the matching degree is converted into a confidence score, which intuitively reflects the reliability of each filter in processing the current scene path. Preferably, Euclidean distance or cosine similarity algorithms can be used to calculate the matching degree to improve the accuracy of the evaluation.

[0061] S1302. Normalize the weights of each filter according to the confidence level, and use the normalized filters to filter the at least one planned path based on their own optimization parameters. Then, weight and fuse the filtered paths according to the normalized weights based on the trajectory continuity coefficient to obtain the initial planned path.

[0062] Here, weight normalization can be understood as converting the confidence scores of each filter into weight values ​​that sum to 1. Optimization parameters can be understood as pre-adjusted parameters in the filters to adapt to specific scenarios. Trajectory continuity coefficients can be understood as coefficients used to ensure smooth transitions between paths after fusion. The initial planned path can be understood as the preliminary smooth path obtained after filtering and fusion.

[0063] Specifically, the confidence level is first converted into a reasonable weight allocation, then each filter processes the path according to its own parameters, and finally the path is fused by combining the continuity coefficient to ensure that the initial path is smooth and the connection is natural. Preferably, the trajectory continuity coefficient can be dynamically adjusted according to the path curvature, and the larger the curvature, the higher the coefficient should be.

[0064] S1303. Construct a loss function based on the smoothness evaluation index of the initial planned path and iteratively update the filter parameters and weights. When the smoothness evaluation index meets the preset threshold, end the iteration and determine the smoothed initial planned path as the target planned path.

[0065] Here, the smoothness evaluation index can be understood as an indicator used to measure the smoothness of the path. The loss function can be understood as a function used to measure the difference between the initial planned path and the ideal path. The preset threshold can be understood as a critical value pre-set based on experience to judge whether the path smoothness meets the standard; this embodiment does not impose specific restrictions on it.

[0066] Specifically, the difference between the initial path and the ideal state is quantified using a loss function. Parameters and weights are continuously adjusted to optimize the path until smoothness meets the target, ultimately outputting a usable target path. Preferably, a maximum number of iterations is also set to avoid infinite iterations due to the threshold not being met, thus improving processing efficiency.

[0067] The technical solution of this invention solves the technical problems of inaccurate scene adaptation of multimodal filters and insufficient smoothness after path fusion by calculating the matching degree between scene labels and each filter and normalizing the weights, and iteratively optimizing the parameters based on the smoothness index after the planned path is filtered and fused. It achieves the technical effect of dynamic matching between path and driving scene and smoothness meeting the standard.

[0068] Figure 2b A flowchart of the model path planning process for another method for smoothing autonomous driving paths provided in an embodiment of the present invention; as shown. Figure 2b As shown, the method includes;

[0069] S210. Acquire streaming image sequences and environmental feature information through vehicle-mounted equipment.

[0070] Specifically, the vehicle uses onboard cameras, sensors, and other equipment to capture surrounding images and environmental data in real time, providing raw materials for subsequent path generation and scene classification.

[0071] S220. Input the streaming image sequence and the environmental feature information into the end-to-end autonomous driving model. The end-to-end autonomous driving model outputs at least one planned path through feature extraction, integration of dynamic and static element information and path probability mapping.

[0072] Specifically, after inputting streaming image sequences and environmental feature information into the end-to-end autonomous driving model, the model first extracts visual features from the continuous image sequence, such as lane lines and obstacle outlines, while integrating dynamic elements in the environmental features, such as the motion state of traffic participants, and static elements, such as road curvature and intersection structure. Then, through a path probability mapping algorithm, combined with the driving destination and safety rules, it outputs at least one planned path that meets basic driving requirements.

[0073] The technical solution of this invention collects streaming image sequences and environmental feature information by vehicle-mounted equipment and inputs them into an end-to-end model. Through feature extraction, integration of dynamic and static information and path mapping, a planned path is generated. This solves the technical problems of lack of real-time environmental data support and insufficient feasibility of the initial path in autonomous driving path planning, and achieves the technical effect of reliable initial planned path output based on real scene data.

[0074] As an optional example of Embodiment 1 of the present invention, the smoothing processing of the autonomous driving path in this embodiment further includes:

[0075] Step 1. Image sequence acquisition: Streaming image sequences are acquired using an onboard camera as input.

[0076] Preferably, it can also collect environmental characteristic information.

[0077] Step 2. End-to-end model trajectory prediction: The input is the acquired image data. The feature extractor and BEV encoder in the end-to-end model convert the image information into top-view features. The vectorized motion transformer extracts dynamic target information in the scene, realizing dynamic target detection and vectorized trajectory prediction. The vectorized map transformer extracts vectorized static element information (such as lane lines, curbs, and sidewalks) in the scene. The planning transformer takes implicit dynamic and static features as input, which are integrated into a high-dimensional implicit feature representation (embedding). Information related to planning is extracted from it. The self-attention mechanism of the Transformer captures long-distance dependencies between features. Finally, the encoded features are mapped to the path probability space through a multilayer perceptron to complete the path planning of the autonomous vehicle and generate a candidate path distribution. Preferably, image sequences and environmental feature information can be input.

[0078] Step 2. Context-aware decision-making module. The input of this step is environmental feature information (including traffic participant density and trajectory prediction, road curvature, weather information encoding, traffic light status at intersections, etc.). The module classifies the scene according to road type (highway / city / rural), weather conditions (sunny / rainy / foggy) and traffic flow density. The classification result is the scene label "scenes".

[0079] Step 4. Multimodal Kalman Filter Library: The input to this step is the scene splitting label. The scene matching degree is calculated based on the scene label and the scene features targeted by the filters in the filter library. Specifically, the calculation is performed using a matching function. The matching function can be based on Euclidean distance or cosine similarity, calculating the confidence level of each filter output trajectory point solely based on scene matching degree. ;

[0080]

[0081] Step 5. Dynamic Fusion Module: The confidence level c of each filter in the input multimodal Kalman filter is used in this step. i First, calculate the trajectory after filtering by each filter. Here, k represents time k, and i represents the i-th filter. It should be noted that each filter in the multimodal Kalman filter library is optimized for a specific scenario, configured with different state transition matrices Fi and noise covariance matrices Qi and Ri. The state equation is:

[0082]

[0083] The update equation for the state covariance matrix is:

[0084]

[0085] The filtered trajectory can be output using various filter libraries.

[0086] Then, the confidence weights are normalized to obtain the initial weights:

[0087]

[0088] Finally, the normalized weights are assigned to the corresponding filter output trajectories, and the fused trajectory can be represented as:

[0089]

[0090] in, It is the trajectory continuity coefficient, which ensures that the trajectory does not change too much in consecutive moments.

[0091] Step 6. Feedback Learning Module: This module dynamically adjusts the filter parameters (state transition matrix Fi and noise covariance matrices Qi, Ri) and the trajectory weights output by each filter in the trajectory fusion stage. The input is the filtered and smoothed trajectory. It calculates the Jerk value (rate of change of acceleration), the deviation from the original reference trajectory (PathDeviation (PD), and the smoothness index.

[0092]

[0093] Therefore, the objective function can be defined as:

[0094]

[0095] The filter parameters and fusion weight coefficients are updated using stochastic gradient descent. The weight update formula is as follows:

[0096]

[0097] in, It is the learning rate.

[0098] The technical solution of this invention improves ride comfort by smoothing the planned path directly output by the end-to-end model. Simultaneously, by combining a context-aware decision-making module, it dynamically fuses the weights of each filter to avoid abrupt changes in planning due to scene alterations. Compared to standalone modular path planning methods, this enhances the ability to plan paths in complex scenarios. Furthermore, the feedback learning module supports adaptive adjustments to new scenarios through feedback learning.

[0099] Figure 3 This is a schematic diagram of the structure of an autonomous driving path smoothing device provided in an embodiment of the present invention. Figure 3 As shown, the device includes: an output path acquisition module 310, a scene label determination module 320, and a smoothing processing module 330.

[0100] The output path acquisition module 310 is used to acquire at least one planned path output by the end-to-end model; wherein the planned path is generated by the end-to-end autonomous driving model based on the streaming image sequence and environmental feature information collected by the on-board equipment; the scene label determination module 320 is used to classify the current driving scene based on the environmental feature information and the context perception decision module to determine the scene label; the smoothing processing module 330 is used to call a preset multimodal Kalman filter library to smooth at least one planned path based on the scene label to obtain the target planned path.

[0101] The technical solution of this invention involves obtaining at least one planned path output by an end-to-end model. This planned path is generated by the end-to-end autonomous driving model based on streaming image sequences and environmental feature information collected by onboard equipment. The current driving scenario is classified based on the environmental feature information and a context-aware decision module to determine scenario labels. A preset multimodal Kalman filter library is then invoked to smooth the at least one planned path based on the scenario labels to obtain the target planned path. This solves the technical problems of insufficient smoothness and poor scenario adaptability of the end-to-end model output path in complex scenarios, achieving the technical effect of improving the safety and ride comfort of autonomous driving paths.

[0102] In some alternative embodiments, the smoothing module includes:

[0103] The confidence determination unit is used to call a preset multimodal Kalman filter library and at least one planning path to calculate the matching degree between the scene label and each filter and determine the corresponding confidence degree.

[0104] An initial path determination unit is used to normalize the weights of each filter according to the confidence level, filter the at least one planned path through each normalized filter based on its own optimization parameters, and fuse the filtered paths according to the normalized weights based on the trajectory continuity coefficient to obtain the initial planned path.

[0105] The target path determination unit is used to construct a loss function based on the smoothness evaluation index of the initial planned path, iteratively update the filter parameters and weights, and end the iteration when the smoothness evaluation index meets a preset threshold, and determine the smoothed initial planned path as the target planned path.

[0106] In some alternative embodiments, the environmental feature information includes at least traffic participant density, road curvature, weather conditions, and intersection traffic signals.

[0107] In some optional embodiments, the scene label determination module includes:

[0108] The feature extraction unit is used to extract features from the environmental feature information through the context-aware decision module to obtain traffic flow density features, road structure features, weather state features, and traffic signal features.

[0109] The scene segmentation unit is used to segment the traffic flow density feature, the road structure feature and the weather state feature based on preset scene classification rules, so as to obtain traffic level classification results, road type classification results and weather state classification results.

[0110] The label determination unit is used to take the combined result of the traffic level classification result, the road type classification result and the weather state classification result as the classification of the current driving scenario, and generate a scenario label containing the combined result.

[0111] In some alternative embodiments, the context-aware decision module uses a lightweight decision tree or deep learning classification model to classify the scene.

[0112] In some optional embodiments, the optimization parameters of each filter in the multimodal Kalman filter library include the state transition matrix, the process noise covariance matrix, and the observation noise covariance matrix, and the parameters of different filters are optimized for at least one scenario combination.

[0113] In some alternative embodiments, the apparatus further includes:

[0114] The data acquisition module is used to acquire streaming image sequences and environmental feature information through onboard equipment before obtaining at least one planned path from the end-to-end model output.

[0115] The path output module is used to input the streaming image sequence into the end-to-end autonomous driving model. The end-to-end autonomous driving model outputs at least one planned path through feature extraction, integration of dynamic and static element information and path probability mapping.

[0116] The autonomous driving path smoothing device provided in this embodiment of the invention can execute the autonomous driving path smoothing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0117] Figure 4 This is a schematic diagram of an electronic device for implementing the smoothing method for autonomous driving paths according to embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0118] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0119] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0120] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the smoothing processing of autonomous driving paths.

[0121] In some embodiments, the smoothing of the autonomous driving path of the method vehicle can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the smoothing of the autonomous driving path of the method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the smoothing of the autonomous driving path of the method by any other suitable means (e.g., by means of firmware).

[0122] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0124] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0126] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0127] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0128] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0129] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for smoothing autonomous driving paths, characterized in that, include: Obtain at least one planned path output by the end-to-end model; wherein the planned path is generated by the end-to-end autonomous driving model based on the streaming image sequence and environmental feature information collected by the on-board equipment; Based on the environmental feature information and the context-aware decision module, the current driving scenario is classified to determine the scenario label; The preset multimodal Kalman filter library is invoked to smooth at least one planned path based on the scene label in order to obtain the target planned path.

2. The method according to claim 1, characterized in that, The step of calling a preset multimodal Kalman filter library to smooth at least one planned path based on the scene labels to obtain the target planned path includes: The matching degree between the scene label and each filter is calculated by calling a preset multimodal Kalman filter library and at least one planning path, and the corresponding confidence level is determined. The weights of each filter are normalized according to the confidence level. The normalized filters are then used to filter the at least one planned path based on their own optimization parameters. The filtered paths are then weighted and fused according to the normalized weights based on the trajectory continuity coefficient to obtain the initial planned path. Based on the smoothness evaluation index of the initial planned path, a loss function is constructed to iteratively update the filter parameters and weights. When the smoothness evaluation index meets the preset threshold, the iteration ends, and the smoothed initial planned path is determined as the target planned path.

3. The method according to claim 1, characterized in that, The environmental characteristics information includes at least traffic participant density, road curvature, weather conditions, and intersection traffic signals.

4. The method according to claim 1, characterized in that, The classification of the current driving scenario based on the environmental feature information and the context-aware decision module to determine the scenario label includes: The context-aware decision-making module extracts features from the environmental feature information to obtain traffic flow density features, road structure features, weather state features, and traffic signal features. Based on preset scene classification rules, the traffic flow density features, road structure features, and weather state features are classified respectively to obtain traffic level classification results, road type classification results, and weather state classification results. The combined result of the traffic level classification, the road type classification, and the weather condition classification is used as the classification of the current driving scenario, and a scenario label containing the combined result is generated.

5. The method according to claim 1, characterized in that, The context-aware decision-making module uses a lightweight decision tree or deep learning classification model to classify scenes.

6. The method according to claim 1, characterized in that, The optimized parameters of each filter in the multimodal Kalman filter library include the state transition matrix, the process noise covariance matrix, and the observation noise covariance matrix. The parameters of different filters are optimized for at least one scenario combination.

7. The method according to claim 1, characterized in that, Before obtaining at least one planned path from the end-to-end model output, the following is also included: Streaming image sequences and environmental feature information are acquired using vehicle-mounted equipment; The streaming image sequence and the environmental feature information are input into the end-to-end autonomous driving model. The end-to-end autonomous driving model outputs at least one planned path through feature extraction, integration of dynamic and static element information and path probability mapping.

8. A smoothing device for an autonomous driving path, characterized in that, include: The output path acquisition module is used to acquire at least one planned path output by the end-to-end model; wherein, the planned path is generated by the end-to-end autonomous driving model based on the streaming image sequence and environmental feature information collected by the on-board equipment; The scene label determination module is used to classify the current driving scene based on the environmental feature information and the context-aware decision module in order to determine the scene label; The smoothing module is used to call a preset multimodal Kalman filter library to smooth at least one planned path based on the scene label in order to obtain the target planned path.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the smoothing method for the autonomous driving path according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the smoothing method for the autonomous driving path according to any one of claims 1-7.

Images