Automatic driving path planning method and system and medium

Abstract

The invention relates to an automatic driving path planning method and system and a medium, and relates to the technical field of automatic driving, and the method comprises the steps: obtaining multi-modal sensor data; processing the multi-modal sensor data using a spatio-temporal feature extraction network; generating a path planning decision by using a hierarchical deep reinforcement learning model, wherein the hierarchical deep reinforcement learning model comprises a first strategy network for high-level decision and a second strategy network for low-level control; and adjusting the generated path in real time based on a dynamic optimization engine. According to the method, efficient, safe and flexible automatic driving path generation can be realized in a complex dynamic environment, and the adaptability and reliability of an automatic driving system in various scenes are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an autonomous driving path planning method, system, and medium. Background Technology

[0002] With the rapid development of autonomous driving technology, path planning, as one of its core technologies, has received widespread attention. Currently, mainstream path planning methods mainly include search-based methods (such as A*, RRT, etc.), sampling-based methods (such as PRM, etc.), and optimization-based methods (such as MPC, etc.). These methods perform well in static or simple dynamic environments, but still have some limitations in complex dynamic environments. For example, traditional methods struggle to effectively handle sudden obstacles and multi-objective optimization problems, and their computational efficiency often falls short of real-time requirements.

[0003] Existing technologies suffer from the following problems when handling path planning in complex dynamic environments: First, their ability to predict and process dynamic obstacles is limited, making it difficult to cope with unexpected situations; second, the efficiency of multi-sensor data fusion is low, failing to fully utilize multimodal information; third, the coordination between decision-making and control is insufficient, making it difficult to balance discrete decision-making and continuous control; and finally, their generalization ability to extreme scenarios is weak, relying on a large amount of labeled data. These problems limit the performance and safety of autonomous driving systems in complex environments. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide an autonomous driving path planning method, system and medium that can achieve efficient, safe and flexible autonomous driving path generation in complex dynamic environments, effectively improving the adaptability and reliability of autonomous driving systems in various scenarios.

[0005] To achieve the above objectives, the present invention adopts the following technical solution.

[0006] Firstly, the present invention provides an autonomous driving path planning method, which adopts the following technical solution: Acquire multimodal sensor data; The multimodal sensor data is processed using a spatiotemporal feature extraction network; A hierarchical deep reinforcement learning model is used to generate path planning decisions. This model includes a first policy network for high-level decision-making and a second policy network for low-level control. The generated path is adjusted in real time based on a dynamic optimization engine.

[0007] Furthermore, in the above path planning method, the multimodal sensor data includes data from cameras, millimeter-wave radar, and lidar.

[0008] Furthermore, in the above path planning method, the spatiotemporal feature extraction network includes: The spatiotemporal attention layer is used to calculate the association weights between obstacle trajectories and road semantics; Three-dimensional convolutional layers are used to extract spatial features; and Spatiotemporal graph convolutional layers are used to model the interaction between vehicles and obstacles.

[0009] Furthermore, in the above path planning method, the first policy network uses a near-end policy optimization algorithm to generate discrete actions.

[0010] Furthermore, in the above path planning method, the second policy network uses a deep deterministic policy gradient algorithm to generate continuous actions.

[0011] Furthermore, the above path planning method also includes: The hierarchical deep reinforcement learning model is trained using a hierarchical course learning strategy, which includes three stages: static environment, dynamic obstacles, and extreme scenarios.

[0012] Furthermore, in the above path planning method, the dynamic optimization engine includes: The Monte Carlo tree search module is used to quickly generate candidate paths; and The model prediction control module is used to track the optimal path.

[0013] Furthermore, in the above path planning method, the triggering conditions for the dynamic optimization engine include: The environmental uncertainty entropy value is greater than a preset threshold; or The collision time is less than the preset time.

[0014] Furthermore, the above path planning method also includes: The generated path is evaluated using a multi-objective reward function, which includes dynamic weighting of safety, comfort, efficiency, and compliance.

[0015] Furthermore, the above path planning method also includes: The hierarchical deep reinforcement learning model is lightweighted using model pruning and TensorRT acceleration techniques to meet the real-time requirements of automotive embedded platforms.

[0016] Secondly, the present invention provides an autonomous driving path planning system, applied to the path planning method described in any one of the first aspects above, and employs the following technical solution: A multimodal sensor module is used to acquire multimodal sensor data; A spatiotemporal feature extraction module is used to process the multimodal sensor data; A hierarchical deep reinforcement learning module is used to generate path planning decisions; The dynamic optimization module is used to adjust the generated path in real time.

[0017] Thirdly, the present invention provides a readable storage medium, which adopts the following technical solution: A readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in any one of the first aspects above.

[0018] In summary, compared with the prior art, the present invention has at least one of the following beneficial technical effects: This invention integrates multimodal sensor data, a spatiotemporal feature extraction network, a hierarchical deep reinforcement learning model, and a dynamic optimization engine to more effectively handle path planning problems in complex dynamic environments. This approach may improve the prediction and processing capabilities for dynamic obstacles, enhance the efficiency of multi-sensor data fusion, improve the coordination between decision-making and control, and strengthen generalization capabilities in extreme scenarios. Furthermore, the introduction of the dynamic optimization engine may enable the system to adjust the path in real time, thereby improving the adaptability, safety, and reliability of autonomous driving systems in complex environments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a specific embodiment of the autonomous driving path planning method of the present invention.

[0021] Figure 2 This is a schematic diagram illustrating the implementation principle of the autonomous driving path planning method of the present invention.

[0022] Figure 3 This is a schematic diagram of a specific embodiment of the autonomous driving path planning system of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, it should be understood that the specific embodiments described herein are only for illustration and explanation of this application and are not intended to limit this application.

[0024] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments of this application. Furthermore, the descriptions of each embodiment in the following embodiments have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0025] The method steps described in this embodiment of the invention can be executed in the order described in the specific implementation, or the execution order of each step can be adjusted according to actual needs, provided that the technical problem can be solved. These are not listed one by one here.

[0026] The following is in conjunction with the appendix Figure 1-3 The present invention will be described in further detail below.

[0027] Reference Figure 1 and Figure 2 An autonomous driving path planning method includes the following steps.

[0028] S1, acquire multimodal sensor data.

[0029] Specifically, in the autonomous driving path planning process, acquiring multimodal sensor data includes collecting environmental perception information through various sensors configured on the vehicle. Specifically, the camera system includes a front-facing dual 120° wide-angle lens (1920×1080@30Hz) and a side-facing 60° narrow-angle lens, used to capture lane lines, traffic signs, and obstacle image information; the millimeter-wave radar uses four 77GHz radars, capable of tracking 64 targets within a 200-meter range, primarily used for mid-to-long-range dynamic obstacle detection; the LiDAR is a 16-line rotating type with a 360° horizontal field of view and a point cloud sampling capability of 300,000 points / second, used to provide three-dimensional spatial structure information; the positioning unit uses a combination of high-precision RTK-GPS (horizontal error ≤2cm) and a six-axis IMU (sampling rate 100Hz) to acquire the vehicle's pose information. In addition, to ensure the collaborative processing of sensor data, time synchronization is achieved through a hardware clock based on the PTP protocol (delay ≤ 5ms), and a calibration matrix is ​​used to uniformly map various types of sensor data to the vehicle coordinate system to complete spatial alignment and ensure the spatiotemporal consistency of subsequent feature extraction and decision processing.

[0030] S2, using a spatiotemporal feature extraction network to process the multimodal sensor data.

[0031] Specifically, the multimodal sensor data acquired in step S1 is input into a spatiotemporal feature extraction network for processing to extract key features reflecting dynamic environmental changes and spatial structure. This network first encodes 8 channels and 5 frames of historical data (0.5-second time window, 256×256 spatial resolution), including RGB images, radar reflection intensity, and LiDAR point cloud density. Then, it calculates the temporal and spatial association weights between obstacles and road semantics through an 8-head spatiotemporal cross-attention mechanism, thereby highlighting key areas (such as potential collision paths). Subsequently, the network uses 4 layers of 3D convolutions (3×3×3 kernel, 2×2×2 stride) to extract joint spatial and temporal features, with an output dimension of 512. To further model the dynamic interaction relationships between traffic participants, the network also introduces a 3-layer dynamically updated ST-GCN (Spatiotemporal Graph Convolutional Network), whose adjacency matrix is ​​updated at a frequency of 10Hz, enabling the modeling of the interaction behavior between vehicles and obstacles, thus providing high-quality spatiotemporal feature support for subsequent path planning decisions.

[0032] S3, using a hierarchical deep reinforcement learning model to generate path planning decisions, wherein the hierarchical deep reinforcement learning model includes a first policy network for high-level decision-making and a second policy network for low-level control.

[0033] Specifically, a hierarchical deep reinforcement learning model is used to generate path planning decisions. This model includes a first policy network for high-level decision-making and a second policy network for low-level control. The first policy network uses the Proximal Policy Optimization (PPO) algorithm, taking the global spatiotemporal features extracted in step S2 (dimension 512, including lane topology, traffic light status, and predicted obstacle trajectories) as input, and outputting discrete path planning decisions such as changing lanes left, changing lanes right, keeping the lane, and emergency stopping. The second policy network uses the Deep Deterministic Policy Gradient (DDPG) algorithm, taking local environmental state information (dimension 256, including current vehicle speed, steering angle, and deviation from the lane centerline) as input, and outputting continuous actions for fine-grained control, including steering angle (range -30° to 30°) and acceleration (range -3m / s² to 2m / s²). During training, the second policy network uses OU noise for exploration, with an initial standard deviation σ=0.2, which gradually decreases as training progresses, thereby achieving efficient coordination between high-level path selection and low-level control commands.

[0034] S4 uses a dynamic optimization engine to adjust the generated paths in real time.

[0035] Specifically, the dynamic optimization engine adjusts the path generated by the hierarchical deep reinforcement learning model in real time. The engine triggers replanning operations by sensing environmental uncertainties and potential collision risks. Specifically, if the environmental uncertainty entropy predicted by the Bayesian neural network exceeds 0.7 or the time-to-collision (TTC) between the vehicle and an obstacle is less than 1.5 seconds, the path replanning mechanism is triggered. In this case, the dynamic optimization engine first expands by 100 nodes in each control cycle using the MCTS (Monte Carlo Tree Search) algorithm, simulating a 10-step path (approximately 0.5 seconds) to quickly generate multiple candidate paths. Then, it uses a trained policy network to evaluate the safety of each path using a Q-value and selects the optimal path. Finally, the MPC (Model Predictive Control) module performs trajectory smoothing tracking at 50ms intervals, using a weight matrix Q=diag(1,1,0.5) for error constraints to ensure real-time and smooth path adjustments, with the overall end-to-end response latency controlled within 50ms.

[0036] The path planning method described in this invention achieves millisecond-level response and precise avoidance of sudden obstacles in complex dynamic traffic environments through high-precision acquisition of multimodal sensor data, deep extraction of spatiotemporal features, path decision generation by a hierarchical deep reinforcement learning model, and real-time adjustment by a dynamic optimization engine. This method not only improves the accuracy and stability of path planning but also significantly enhances the system's generalization ability in extreme weather and unstructured scenarios. It has advantages such as low collision rate, low latency, and high comfort, meeting the comprehensive requirements of L4 autonomous driving for real-time performance, safety, and passenger experience.

[0037] Furthermore, as one embodiment of the present invention, the spatiotemporal feature extraction network includes: The spatiotemporal attention layer is used to calculate the association weights between obstacle trajectories and road semantics; the 3D convolutional layer is used to extract spatial features; and the spatiotemporal graph convolutional layer is used to model the interaction between vehicles and obstacles.

[0038] Specifically, the spatiotemporal attention layer performs temporal alignment and spatial feature focusing on historical frame data from multimodal sensors. It introduces a multi-head attention mechanism to uncover key correlation regions between obstacle trajectories and road semantics. The input is a sequence of feature maps within a 5-frame time window, each frame being 256×256×8 pixels. Temporal information is first added through positional encoding, then the data is fed into an 8-head spatiotemporal cross-attention unit to calculate the attention distribution in both the temporal and spatial dimensions. In the specific calculation, a standard scaled dot product attention mechanism is used, with the following formula: ; Where Q, K, and V are the query, key, and value matrices, respectively, obtained from the input features through linear transformation. Let the dimension of the key vector be . The introduced time difference coding term enhances the ability to perceive the sequence of events between different time steps, thereby mitigating information distortion caused by sensor asynchrony. This attention mechanism strengthens the influence of the predicted region (such as a pedestrian about to cross) on the current path planning and visualizes it as a heatmap through weight distribution to highlight key obstacles and lane areas, thus improving the accuracy and efficiency of spatiotemporal feature extraction.

[0039] The 3D convolutional layer is used to extract joint features in both spatial and temporal dimensions from the fused multimodal temporal data, fully preserving the dynamic changes in obstacle trajectories and environmental structural information. The input is a sequence of multi-frame feature maps output from the spatiotemporal attention layer, with dimensions of 5×256×256×C, where 5 represents the number of time frames, 256×256 represents the spatial resolution, and C represents the number of channels. This module consists of four stacked 3D convolutional layers, each with a kernel size of 3×3×3 and a stride of 2×2×2. The ReLU activation function is used, followed by batch normalization (BatchNorm) after each layer to accelerate convergence and improve stability. The calculation formula is as follows: ; in, For the input feature map, Here, b represents the 3D convolution kernel weights, and b represents the bias term. Representing time and spatial location, 𝑐 is the input channel index. Through this 3D convolutional processing, the network can extract spatiotemporal joint features such as vehicle following behavior and dynamic obstacle movement trends, laying the foundation for subsequent spatiotemporal graph convolutional networks to model the interaction relationships between entities.

[0040] The spatiotemporal graph convolutional layer is used to model the dynamic interaction between vehicles and obstacles. It adopts an ST-GCN (Spatial-Temporal Graph Convolutional Network) architecture based on a dynamic graph structure. The input is the spatiotemporal features extracted by the 3D convolutional layer, constructing a graph structure G=(V, E), where the node set V represents the traffic participants at the current time (such as the vehicle itself, the vehicle in front, pedestrians, etc.), and the edge set E represents their spatiotemporal relationships. The calculation formula for each graph convolution operation is as follows: ; in, For the first Layer node feature representation, It is an adjacency matrix with self-loops. It is the corresponding degree matrix. To learn the weight parameters, This is the ReLU activation function. To model the evolutionary characteristics over time, the adjacency matrix is ​​updated at each time step. It reflects the spatial proximity and velocity direction consistency among traffic participants across different time frames, with an update frequency of 10Hz. This module dynamically captures interaction behavior patterns (such as following and avoiding) through three stacked graph convolutional layers, providing the policy network with highly robust spatiotemporal interaction features.

[0041] Furthermore, as one embodiment of the present invention, the first policy network employs a near-end policy optimization algorithm to generate discrete actions.

[0042] Specifically, the first policy network employs the Proximal Policy Optimization (PPO) algorithm to generate discrete actions during autonomous driving, including high-level path planning decisions such as changing lanes to the left, changing lanes to the right, maintaining lane position, and emergency braking. This network takes the global spatiotemporal state vector extracted in steps S2 and S3 as input, with a feature dimension of 512, covering information such as lane topology, traffic signal status, and predicted obstacle trajectories. During training, a shearing probability ratio objective function is used to limit the update magnitude between the old and new policies, avoiding excessive policy shift. The specific optimization objective is: ; in, It is the strategy probability ratio. For the dominant function, The shearing coefficient is 0.2 in this embodiment. This structure effectively improves the stability of policy updates. The training process uses GAE (Generalized Advantage Estimation) to alleviate the high variance problem, enabling the policy network to quickly generate robust high-level discrete path decisions in complex scenarios, with an average success rate of over 98%.

[0043] Furthermore, as one embodiment of the present invention, the second policy network employs a deep deterministic policy gradient algorithm to generate continuous actions.

[0044] Specifically, the second policy network employs the Deep Deterministic Policy Gradient (DDPG) algorithm to generate continuous action control commands during autonomous driving, including fine-grained steering angle control (range -30° to 30°) and acceleration adjustment (range -3m / s² to 2m / s²), to achieve path tracking and vehicle control. The network input consists of local state information with a feature dimension of 256, covering vehicle dynamics such as current speed, steering angle, and deviation from the lane centerline. During training, the policy network and the evaluation network (Q-network) co-optimize, aiming to maximize the action value function under the current policy. ; in, This represents the action output by the policy network. , The parameters are those of the policy and the Q-network, respectively. To improve exploration efficiency, the Ornstein-Uhlenbeck (OU) noise mechanism is used as the motion perturbation source, with an initial standard deviation σ set to 0.2, which gradually decreases during training. Furthermore, a soft update strategy (τ=0.001) is used to synchronize the target network, improving training stability. This network can continuously output high-precision continuous control commands in complex dynamic scenarios, with the average lateral error controlled within 0.3 meters, effectively enhancing the stability and safety of vehicle driving.

[0045] Furthermore, as one embodiment of the present invention, the path planning method further includes: The hierarchical deep reinforcement learning model is trained using a hierarchical course learning strategy, which includes three stages: static environment, dynamic obstacles, and extreme scenarios.

[0046] Specifically, the path planning method employs a hierarchical course learning strategy to progressively train a hierarchical deep reinforcement learning model, thereby improving its policy generalization ability and convergence efficiency in scenarios of varying complexity. This strategy divides the training process into three stages: the first stage is static environment training, where the vehicle completes the path planning task without dynamic obstacles, and proceeds to the next stage once the success rate reaches over 99%; the second stage is dynamic obstacle training, where vehicles and pedestrians with speeds ranging from 0 to 15 m / s are introduced into the training scenario, and the reinforcement learning model acquires the ability to predict and avoid dynamic targets, with the target metric being a multi-target tracking accuracy (MOTA) of over 85%; the third stage is extreme scenario training, including path planning in low visibility conditions (such as heavy rain or nighttime) and complex road structures (such as irregular intersections or unsignalized junctions).

[0047] During training, exploration parameters are dynamically adjusted. For example, the standard deviation σ of OU noise is set to 0.2 in the initial stage and gradually decayed to 0.05 as the stage progresses, in order to improve stability and policy refinement. This hierarchical course learning strategy significantly accelerates the model convergence speed (by about 30%) and ensures stable model performance in long-tailed complex scenarios, with a path planning success rate of over 90%.

[0048] Furthermore, as one embodiment of the present invention, the dynamic optimization engine includes: a Monte Carlo tree search module for quickly generating candidate paths; and a model prediction control module for tracking the optimal path.

[0049] Specifically, the dynamic optimization engine is composed of a Monte Carlo Tree Search (MCTS) module and a Model Predictive Control (MPC) module, used to optimize and adjust path planning results in real time under complex dynamic environments. The MCTS module expands the search by 100 nodes in each control cycle, simulating a depth of 10 steps (corresponding to approximately 0.5 seconds of driving behavior). Based on the Upper Confidence Bound (UCB) strategy, it selects the optimal path branch and quickly generates multiple feasible candidate paths. Subsequently, the MPC module uses the selected optimal path as the tracking target and employs an OSQP quadratic programming solver for trajectory smoothing control. The control cycle is 50ms, and the error weight matrix is ​​set to Q=diag(1,1,0.5), corresponding to the adjustment intensity of lateral position, longitudinal position, and speed errors, respectively. The continuous control commands output by this module can accurately drive the vehicle to execute the current optimal path, thereby ensuring that the autonomous driving system has an end-to-end latency of ≤50ms, effectively coping with highly dynamic scenarios such as sudden obstacles and path conflicts.

[0050] Furthermore, as an embodiment of the present invention, the triggering conditions of the dynamic optimization engine include: the environmental uncertainty entropy value is greater than a preset threshold; or the collision time is less than a preset time.

[0051] Specifically, the triggering conditions for the dynamic optimization engine include real-time assessment of environmental uncertainties and potential collision risks to determine whether to activate the path replanning mechanism. Environmental uncertainty is modeled using a Bayesian neural network to assess the confidence level of the perceived data, calculating the entropy value of its predicted output as an uncertainty measure. When this entropy value exceeds a preset threshold (e.g., 0.7), it is considered that the current environment has significant perceptual ambiguity or dynamic changes, requiring the activation of the optimization engine. Simultaneously, the system also calculates the time-to-collision (TTC) between the vehicle and obstacles in front in real time. If it is less than a set threshold (e.g., 1.5 seconds), it is determined that there is a potential collision risk, and the dynamic optimization engine will be activated immediately. Through this dual triggering mechanism, the system can respond rapidly when perception is unstable or danger is imminent, performing path replanning and trajectory tracking control, thereby ensuring the safety and robustness of autonomous vehicles in complex dynamic scenarios.

[0052] Furthermore, as one embodiment of the present invention, the path planning method further includes: The generated path is evaluated using a multi-objective reward function, which includes dynamic weighting of safety, comfort, efficiency, and compliance.

[0053] Specifically, the path planning method comprehensively evaluates the generated path through a multi-objective reward function to achieve a dynamic balance optimization among safety, comfort, efficiency, and compliance. This reward function uses a weighted summation form, with different sub-reward items corresponding to different objectives. The safety component incorporates a collision risk field model, dynamically assessing potential collision risks based on the relative distance and speed between the vehicle and obstacles. The comfort component constrains vehicle acceleration (rate of change of acceleration), penalizing actions exceeding 1.5 m / s³. The efficiency component rewards based on the travel time along the current path, with shorter times yielding higher rewards. The compliance component sets penalties based on whether traffic signals and lane markings are followed; for example, detecting lane violations or running red lights deducts rewards. The weights of each objective are dynamically adjusted according to the current scenario. For example, efficiency weights are increased in scenarios with high efficiency requirements, such as merging onto highways, while safety and compliance weights are enhanced in complex urban intersections, thus achieving a site-specific multi-objective path optimization strategy.

[0054] Furthermore, as one embodiment of the present invention, the path planning method further includes: The hierarchical deep reinforcement learning model is lightweighted using model pruning and TensorRT acceleration techniques to meet the real-time requirements of automotive embedded platforms.

[0055] Specifically, to meet the stringent requirements of automotive embedded platforms for computing resources and response speed, the path planning method performs lightweight processing on the hierarchical deep reinforcement learning model, employing model pruning and TensorRT acceleration techniques. Model pruning evaluates the importance of weights in each layer of the neural network, gradually removing redundant neurons and connections with minimal impact on decision-making, reducing the overall number of parameters by approximately 50% and lowering computational complexity without significantly reducing model accuracy. Subsequently, using NVIDIA's TensorRT deep learning inference optimization tool, the simplified model undergoes optimization operations such as graph structure fusion, hierarchical reordering, and low-precision (FP16 or INT8) quantization to further improve inference efficiency. The optimized model can be deployed on mainstream automotive-grade embedded chip platforms such as NVIDIA Orin X, with memory usage not exceeding 8GB in actual operation and single inference latency controlled within 10ms, ensuring that the end-to-end response time of the entire path planning system is ≤50ms, fully meeting the real-time requirements of L4 autonomous driving.

[0056] In summary, the path planning method proposed in this invention significantly improves the path planning capability of autonomous driving systems in complex dynamic environments through a collaborative architecture of multimodal sensor data acquisition, spatiotemporal feature extraction, hierarchical deep reinforcement learning decision-making, and dynamic optimization control.

[0057] Table 1 Referring to Table 1, simulation verification shows that this method achieves significant improvements over the traditional A*+MPC scheme in several key performance indicators: the collision rate per unit mileage is reduced from 23 times / 10,000 km to 2 times / 10,000 km, a decrease of 91%; the average system response latency is reduced from 120ms to 50ms, a decrease of 58%; vehicle acceleration is optimized from 2.8m / s³ to 1.5m / s³, improving comfort by 46%; and the multi-target tracking accuracy (MOTA) is improved from 76.2% to 89.7%, an increase of 17.7%. Furthermore, it maintains over 85% perception accuracy in extreme scenarios such as rain and nighttime, with a path planning success rate exceeding 90% and a traffic violation rate of less than 0.1 times / 1,000 km.

[0058] Based on this, the path planning method not only improves safety, comfort and efficiency, but also significantly enhances the autonomous driving system's adaptability and generalization ability to complex environments, meeting the comprehensive technical requirements of L4 autonomous driving for real-time performance and reliability.

[0059] This invention also discloses an autonomous driving path planning system.

[0060] Reference Figure 3An autonomous driving path planning system includes a multimodal sensor module 1, a spatiotemporal feature extraction module 2, a hierarchical deep reinforcement learning module 3, and a dynamic optimization module 4.

[0061] The multimodal sensor module 1 is used for comprehensive perception of the vehicle's surrounding environment, integrating various heterogeneous sensors to achieve high-precision and robust environmental data acquisition. Specifically, it includes: a front-facing dual 120° wide-angle camera (1920×1080@30Hz) and a side-facing 60° narrow-angle camera to capture lane lines, traffic signs, and obstacle image information; four 77GHz millimeter-wave radars with a detection range of 200 meters and the ability to track up to 64 dynamic targets, suitable for long-distance obstacle monitoring; a 16-line 360° rotating lidar (LiDAR) providing high-density point cloud data of 300,000 points per second for constructing a detailed 3D environment model; and a high-precision positioning unit composed of RTK-GPS and a 6-axis IMU, where the GPS horizontal error is less than 2cm and the IMU sampling rate is 100Hz, ensuring the stability of vehicle attitude and position estimation. The module synchronizes the hardware clock via the PTP (Precision Time Protocol) to achieve a sensor data latency of ≤5ms. At the same time, it combines a spatial calibration matrix to uniformly map all sensor data to the vehicle coordinate system, providing spatiotemporally consistent and accurate multimodal input data for subsequent modules.

[0062] The spatiotemporal feature extraction module 2 performs deep processing on the data collected by the multimodal sensor module 1, extracting key spatiotemporal features reflecting environmental structure and dynamic changes. The module input is a multimodal feature tensor consisting of five consecutive frames with a spatial resolution of 256×256 and eight channels, containing information such as RGB images, radar reflection intensity, and LiDAR point cloud density. This module first employs an 8-head cross-attention mechanism through a spatiotemporal attention layer, combined with a temporal difference coding function, to calculate the spatiotemporal correlation weights between obstacle trajectories and road semantics, highlighting key areas (such as potential collision paths). Subsequently, a four-layer 3D convolutional network (3×3×3 kernel, 2×2×2 stride) jointly extracts structured features in both spatial and temporal dimensions, outputting a feature vector of dimension 512. Finally, a three-layer spatiotemporal graph convolutional network (ST-GCN) dynamically constructs a relationship graph between traffic participants, with the adjacency matrix updated at a frequency of 10Hz, used to model the interaction behavior patterns between vehicles and obstacles. This module comprehensively improves the modeling capability for complex dynamic environments and provides high-quality spatiotemporal feature input for subsequent path planning decisions.

[0063] The hierarchical deep reinforcement learning module 3 is used to generate autonomous driving path planning decisions based on spatiotemporal features. It adopts a high-low layered structure, including a first policy network for high-level decision-making and a second policy network for low-level control. The first policy network uses the Proximal Policy Optimization (PPO) algorithm, taking the global state vector (dimension 512) output by the spatiotemporal feature extraction module 2 as input and outputting discrete actions such as lane change to the left, lane change to the right, lane keeping, and emergency braking. During training, a shear probability ratio optimization objective is used to improve the stability of policy updates. The second policy network uses the Deep Deterministic Policy Gradient (DDPG) algorithm, taking the current local state of the vehicle (dimension 256) as input and outputting continuous control commands such as steering angle (range -30° to 30°) and acceleration (range -3m / s² to 2m / s²). To improve exploration efficiency, an OU noise mechanism is introduced and exploration noise is gradually attenuated. A soft update strategy is used during training to maintain the stability of the target network. This module, through high-low layer decoupling, balances discrete path planning and continuous control execution, achieving safe, comfortable, and efficient generation of autonomous driving behavior policies.

[0064] The dynamic optimization module 4 is used to adjust and enhance the safety of the path planning results generated by the hierarchical deep reinforcement learning module 3 in real time. It includes a Monte Carlo Tree Search (MCTS) module and a Model Predictive Control (MPC) module. The dynamic optimization process is triggered when the system detects an environmental uncertainty entropy value greater than 0.7 (evaluated by a Bayesian neural network) or a collision time of less than 1.5 seconds. First, the MCTS module starts from the current state and expands the search by 100 search nodes in each control cycle, simulating 10 future actions. Based on the UCB strategy, it evaluates the potential risks and benefits of candidate paths and generates multiple feasible paths. Then, the MPC module performs trajectory tracking and smooth control on the selected path at 50ms intervals using an OSQP solver, employing an error weight matrix Q=diag(1,1,0.5) to constrain lateral, longitudinal, and velocity deviations. This module can achieve millisecond-level path replanning, with an end-to-end response latency of ≤50ms, effectively improving the reaction speed and operational stability of autonomous driving in the event of emergencies.

[0065] In summary, the autonomous driving path planning system, composed of a multimodal sensor module 1, a spatiotemporal feature extraction module 2, a hierarchical deep reinforcement learning module 3, and a dynamic optimization module 4, achieves efficient path planning and real-time control in complex and dynamic traffic environments through the organic coordination of perception, understanding, decision-making, and execution. Specifically, the multimodal sensor module 1 collects heterogeneous environmental data, including images, radar, lidar, and positioning information; the spatiotemporal feature extraction module 2 fuses and extracts key dynamic and structural features to improve the understanding of obstacle behavior and road semantics; the hierarchical deep reinforcement learning module 3 employs a high-level PPO and a low-level DDPG policy network to achieve collaborative division of labor between path planning and continuous control; and the dynamic optimization module 4 uses a combination of MCTS and MPC mechanisms to perform path replanning and trajectory smoothing control in high-risk or sudden situations. The system as a whole possesses beneficial technical effects such as accurate path generation, low response latency, low collision rate, and high comfort, and maintains robustness in extreme scenarios such as rain, nighttime, and irregular intersections, meeting the comprehensive technical requirements of L4 autonomous driving for safety, real-time performance, and generalization capabilities.

[0066] This invention also discloses a readable storage medium.

[0067] A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any of the above embodiments. The computer-readable storage medium may include any entity or device capable of carrying a computer program, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc. The computer program includes computer program code. The computer program code may be in the form of source code, object code, an executable file, or some intermediate form, etc. The computer-readable storage medium may include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0068] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0069] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a system including a processing module or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0070] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An autonomous driving path planning method, characterized in that, include: Acquire multimodal sensor data; The multimodal sensor data is processed using a spatiotemporal feature extraction network; A hierarchical deep reinforcement learning model is used to generate path planning decisions. The hierarchical deep reinforcement learning model includes a first policy network for high-level decision-making and a second policy network for low-level control. as well as The generated path is adjusted in real time based on a dynamic optimization engine.

2. The path planning method according to claim 1, characterized in that, The multimodal sensor data includes data from cameras, millimeter-wave radar, and lidar.

3. The path planning method according to claim 1, characterized in that, The spatiotemporal feature extraction network includes: The spatiotemporal attention layer is used to calculate the association weights between obstacle trajectories and road semantics; Three-dimensional convolutional layers are used to extract spatial features; and Spatiotemporal graph convolutional layers are used to model the interaction between vehicles and obstacles.

4. The path planning method according to claim 1, characterized in that, The first policy network employs a near-end policy optimization algorithm to generate discrete actions.

5. The path planning method according to claim 1, characterized in that, The second policy network employs a deep deterministic policy gradient algorithm to generate continuous actions.

6. The path planning method according to claim 1, characterized in that, Also includes: The hierarchical deep reinforcement learning model is trained using a hierarchical course learning strategy, which includes three stages: static environment, dynamic obstacles, and extreme scenarios.

7. The path planning method according to claim 1, characterized in that, The dynamic optimization engine includes: The Monte Carlo tree search module is used to quickly generate candidate paths; and The model prediction control module is used to track the optimal path.

8. The path planning method according to claim 7, characterized in that, The triggering conditions for the dynamic optimization engine include: The environmental uncertainty entropy value is greater than a preset threshold; or The collision time is less than the preset time.

9. The path planning method according to claim 1, characterized in that, Also includes: The generated path is evaluated using a multi-objective reward function, which includes dynamic weighting of safety, comfort, efficiency, and compliance.

10. The path planning method according to claim 1, characterized in that, Also includes: The hierarchical deep reinforcement learning model is lightweighted using model pruning and TensorRT acceleration techniques to meet the real-time requirements of automotive embedded platforms.

11. An autonomous driving path planning system, applied to the path planning method as described in any one of claims 1-10, characterized in that, The system includes: A multimodal sensor module is used to acquire multimodal sensor data; A spatiotemporal feature extraction module is used to process the multimodal sensor data; A hierarchical deep reinforcement learning module is used to generate path planning decisions; The dynamic optimization module is used to adjust the generated path in real time.

12. A readable storage medium, characterized in that, The readable storage medium stores computer instructions that, when executed by a processor, implement the path planning method as described in any one of claims 1-10.

Images