Automatic driving path optimization method based on deep reinforcement learning
By employing deep reinforcement learning and a multi-vehicle path conflict detection mechanism, the problem of path conflicts in multi-vehicle scenarios was solved, enabling safe collaboration and adaptive optimization of autonomous vehicles and improving the stability and efficiency of traffic flow.
Patent Information
- Application Number
- CN202511672284.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-14
- Publication Date
- 2026-02-27
AI Technical Summary
Existing autonomous driving path planning methods lack modeling and prediction of the path intentions of other vehicles in multi-vehicle scenarios, leading to path conflicts and reduced traffic efficiency. Furthermore, traditional reinforcement learning models are unstable in high-density traffic flow, making it difficult to achieve global safety assurance.
By employing a deep reinforcement learning-based approach, environmental information is collected in real time through vehicle-mounted sensors and vehicle-to-everything (V2X) systems to construct a dynamic state space. Combining convolutional neural networks and recurrent neural networks, path intent is predicted. A multi-objective reward function and conflict detection mechanism are introduced to achieve multi-vehicle collaborative path planning and conflict avoidance.
It improves the stability and traffic flow efficiency of multi-vehicle collaborative path planning, reduces the probability of path conflicts, and realizes the functional leap from single-vehicle optimization to group intelligent collaboration, possessing environmental adaptability and global coordination capabilities.
Smart Images
Figure CN121583134A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of driving path optimization, in particular to an automatic driving path optimization method based on deep reinforcement learning. BACKGROUND
[0002] With the rapid development of artificial intelligence and intelligent transportation systems, autonomous driving technology has become a hot direction of global technology and industry competition. In the autonomous driving system, path planning is one of the core modules to achieve autonomous driving, and its main task is to plan a safe, efficient and comfortable driving path for the vehicle according to the current traffic environment and vehicle state. Traditional path planning methods mainly rely on pre-set rules or static map information, which is difficult to cope with complex and variable actual traffic environment. Although deep reinforcement learning brings intelligent strategy generation capability to autonomous driving path planning, existing methods still have obvious defects in multi-vehicle scenarios. First, most existing algorithms focus on single-vehicle path optimization, only realizing obstacle avoidance or energy-saving control at the individual decision-making level, lacking modeling and prediction of path intentions of other vehicles, so it is difficult to effectively deal with potential path crossing and conflicts in multi-vehicle mixed driving scenarios. Second, in existing multi-agent reinforcement learning research, collaborative control is usually achieved through sharing of state space or joint reward mechanism, but this approach is prone to unstable learning process in high-density traffic flow, and even causes competitive game among agents, leading to decline in overall traffic efficiency. Third, existing reinforcement learning frameworks rely on static data sets in the training phase, without introducing path conflict penalty constraints or intention reconciliation mechanisms, so the model is prone to local optimal decisions in real environment, lacking global safety guarantee.
[0003] The root cause of the above problems in existing autonomous driving systems is that traditional reinforcement learning models only focus on individual profit maximization, ignoring the dynamic interaction characteristics and path conflict risks among multiple vehicles. When multiple autonomous vehicles are simultaneously in congested, converging or lane-changing dense road sections, if there is a lack of path intention prediction and conflict detection mechanism, the system will be difficult to identify potential collision risks or traffic congestion in advance. This local independent decision often leads to mutual exclusion of strategies among vehicles, such as multiple vehicles simultaneously choosing similar paths or lane-changing operations, thereby causing "path resonance" or "decision shock" phenomena, causing local traffic flow disorder. In addition, if there is a lack of conflict penalty mechanism in the reinforcement learning training process, the model may tend to choose paths with higher short-term rewards but higher potential risks, leading to abnormal behaviors such as sudden stopping, path repeated adjustment and even traffic interference in the execution phase of the vehicle. These abnormal effects not only affect the stability and safety of the autonomous driving system, but also disrupt the coordination of the overall traffic flow, and in severe cases even cause traffic congestion or safety accidents. Therefore, an improved method that combines reinforcement learning path optimization and multi-vehicle conflict detection and reconciliation mechanisms is urgently needed to realize safe collaboration and adaptive optimization of path decisions among multiple vehicles. SUMMARY
[0004] In view of the deficiencies of the prior art, the application provides an automatic driving path optimization method based on deep reinforcement learning, which solves the problems mentioned in the background art.
[0005] To achieve the above object, the application is implemented by the following technical scheme: an automatic driving path optimization method based on deep reinforcement learning, comprising the following steps: collecting state information of the environment in which the automatic driving vehicle is located in real time through a vehicle-mounted sensor and a vehicle networking system, and constructing a dynamic environment state space; on this basis, establishing a reinforcement learning agent based on a deep neural network, outputting a path optimization action according to the current environment state, and continuously optimizing the path planning strategy through online training; in a multi-vehicle cooperative operation environment, calculating the path crossing risk coefficient between any two vehicles based on the predicted path intention of each vehicle; the agent dynamically adjusts the vehicle action strategy according to the conflict intensity and the chronological order; the training process of the agent uses a multi-objective reward function, comprehensively considers the path length, travel time, obstacle avoidance distance, energy consumption and passenger comfort, and dynamically adjusts the reward weight; an experience replay mechanism and a double network structure are introduced in the training process, including a target network and an evaluation network, to improve the training stability and reduce the strategy shock; in the path optimization execution phase, the agent outputs an optimal action sequence according to the current environment state, including lane changing, accelerating, decelerating, detouring and parking waiting operations, to realize real-time path optimization control of the automatic driving vehicle.
[0006] Preferably, the deep neural network adopts a structure combining a convolutional neural network and a recurrent neural network, wherein the convolutional neural network is used to extract the spatial distribution features of road images and obstacles, the recurrent neural network is used to capture the time sequence features of traffic states and vehicle behaviors, and an attention mechanism is introduced to dynamically adjust the environment feature weight, thereby enhancing the perception and path decision-making ability of the agent in complex environments.
[0007] Preferably, a dynamic environment prediction module is set in the path optimization process, which predicts the future traffic state change trend based on historical traffic data and current perception information, and combines a graph neural network to model the traffic flow, accident probability and road traffic capacity, so as to improve the forward-looking and response ability to sudden situations of path planning.
[0008] Preferably, the path optimization method supports multi-vehicle cooperative path planning, which enables multiple automatic driving vehicles to make cooperative decisions in complex traffic environments through sharing of state spaces and joint reward mechanisms, and realizes path coordination and conflict avoidance.
[0009] Preferably, the path optimization method further comprises a multi-vehicle path intention conflict detection and reconciliation mechanism, by constructing a path intention prediction model and a conflict detection function, the intersection probability and time overlap interval between the predicted paths of different vehicles are detected, when the conflict risk coefficient exceeds the preset threshold, the path reconciliation module is triggered, the agent dynamically adjusts the vehicle action strategy according to the conflict intensity and distance difference, including deceleration, detour, lane reservation operations, thereby realizing path coordination and safe avoidance among multiple vehicles, and improving the overall stability and traffic flow efficiency of multi-vehicle cooperative path planning.
[0010] Preferably, a curriculum learning strategy is introduced during training, and the training difficulty is gradually increased from low to high according to the environmental complexity, so that the agent gradually masters the path optimization ability from simple to complex scenes, and improves the model generalization performance and adaptability.
[0011] Preferably, the path optimization strategy is linked with a high-precision map and a V2X communication system, external data such as traffic signal light states, front accident information and construction section positions are obtained in real time, and are used as part of the state input to improve the accuracy, safety and response speed of path decision; the agent predicts the red-green light switching time according to the traffic signal, and adjusts the vehicle speed or path in advance to realize no-stopping through the intersection, improve the traffic efficiency and reduce the energy consumption.
[0012] Preferably, the path optimization method is deployed in the edge computing unit or cloud server of the automatic driving system, supports real-time online updating and offline model iteration, is suitable for L2 to L4 level automatic driving vehicles, and can realize efficient, safe and energy-saving path planning and control in different driving scenes; the system supports a model hot updating mechanism, can complete algorithm version upgrade without interrupting vehicle operation, and ensures that the path optimization strategy continuously adapts to changes in road environment and traffic rules.
[0013] The application provides an automatic driving path optimization method based on deep reinforcement learning, which has the following beneficial effects: (1) When the system is running, the state information of the environment where the autonomous vehicle is located is collected in real time through the vehicle-mounted sensor and the Internet of Vehicles system, and a dynamic environment state space is constructed; on this basis, a reinforcement learning agent based on deep neural network is established, which outputs path optimization actions according to the current environment state, and continuously optimizes the path planning strategy through online training; in the multi-vehicle cooperative operation environment, the path crossing risk coefficient between any two vehicles is calculated based on the predicted path intention of each vehicle; the agent dynamically adjusts the vehicle action strategy according to the conflict intensity and time sequence; the training process of the agent uses a multi-objective reward function, which comprehensively considers the path length, travel time, obstacle avoidance distance, energy consumption and passenger comfort, and dynamically adjusts the reward weight; the experience replay mechanism and double network structure are introduced in the training process, including the target network and the evaluation network, to improve the training stability and reduce the strategy shock; in the path optimization execution stage, the agent outputs the optimal action sequence according to the current environment state, including lane changing, accelerating, decelerating, detouring and parking waiting operations, to realize real-time path optimization control of the autonomous vehicle.
[0014] (2) The present application combines deep reinforcement learning with multi-vehicle path conflict detection mechanism to construct an autonomous driving path optimization method with dynamic self-learning, adaptive adjustment and safe cooperation ability. The method is based on the perception information of the vehicle and the Internet of Vehicles data to construct a multi-dimensional dynamic environment state space, and uses a reinforcement learning agent to realize autonomous decision and optimization update of path actions. In the multi-vehicle cooperative operation environment, the system can make path reconciliation and behavior correction before the conflict occurs through the prediction of vehicle path intention and the quantitative analysis of conflict risk, so as to effectively avoid the potential path crossing and decision interference among multiple vehicles. The present application realizes the functional transition from single vehicle path optimization to group intelligence cooperation, completes the unified system construction of path planning, conflict detection, safety reconciliation and strategy self-evolution, and has strong environmental adaptability and global coordination ability.
[0015] (3) Compared with the traditional path planning algorithm and the existing deep reinforcement learning method, the application realizes significant innovation in algorithm structure and system function. The traditional automatic driving path optimization method mainly depends on static map search or rule-based path selection, which is difficult to cope with the variability of dynamic traffic environment. Although the existing reinforcement learning method has certain adaptive ability, it mainly faces single vehicle decision-making and lacks the fusion of multi-vehicle interaction information and conflict constraint control. The application introduces a "multi-vehicle path intention conflict detection and reconciliation mechanism", embeds a conflict penalty term in the reinforcement learning reward function, so that the model can spontaneously avoid risky paths during the training process, and balance path efficiency and driving safety. At the same time, combined with the deep feature extraction ability of convolutional and recurrent neural networks, and the collaborative fusion of curriculum learning and V2X communication, the application constructs a hierarchical decision network architecture, realizing the collaborative optimization of path planning and safety control. This mechanism breaks through the limitations of existing algorithms, enabling the reinforcement learning model to expand from single-agent decision-making to multi-agent safety collaborative decision-making level.
[0016] (4) Through the fusion of the above innovative structure and algorithm mechanism, the application has achieved significant improvement in path planning performance, safety and traffic flow efficiency. The system can respond to external changes with low delay in complex traffic environment, and realize real-time adaptation to road congestion, obstacle appearance and communication delay and other multi-dimensional interference. The introduced conflict detection and penalty mechanism significantly reduces the path resonance and collision probability between multiple vehicles, making the vehicle group motion trajectory more stable and coordinated; while the adaptive curriculum learning and cloud-edge collaborative mechanism improves the generalization ability and learning efficiency of the model, so that it can maintain strategy consistency and decision reliability in different road environments. Compared with the prior art, the application realizes the transformation from "local optimum" to "global safety optimum", not only optimizes the single vehicle driving path, but also significantly improves the orderliness and traffic efficiency of the overall traffic flow through the intention coordination between multiple vehicles, providing a safe and controllable, stable and efficient path planning solution for intelligent transportation systems. BRIEF DESCRIPTION OF DRAWINGS
[0017] Figure 1 The system framework diagram of the application.
[0018] Figure 2 The workflow diagram of the application. DETAILED DESCRIPTION
[0019] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.
[0020] Implementation Process One: Single Vehicle Path Optimization Process Based on Deep Reinforcement Learning Step One: Environment Information Collection and State Construction First, the autonomous vehicle collects real-time environmental information through on-board cameras, laser radars, millimeter wave radars, and GPS positioning systems. These include road structures (such as lane lines, intersections), traffic flow (such as vehicle density), obstacle distribution (such as pedestrians, stationary vehicles), and the current position and target path of the vehicle. The collected data is input into the state construction module, which converts the raw data into a structured state space as the input of the reinforcement learning agent.
[0021] Step Two: Building a Deep Reinforcement Learning Model Next, a path optimization model based on deep reinforcement learning is built. This model uses a combination of convolutional neural networks (CNN) and recurrent neural networks (RNN), where CNN is used to extract spatial features of road images and obstacles, and RNN is used to capture time series features of traffic states. The model also introduces an attention mechanism, allowing the agent to dynamically focus on environmental features that have a greater impact on current path decisions, improving the flexibility and adaptability of the model.
[0022] Step Three: Designing a Multi-Objective Reward Function To guide the agent to learn the optimal path strategy, a reward function that considers multiple factors is designed. This reward function includes path length, travel time, obstacle avoidance distance, energy consumption, and passenger comfort, among other indicators. For example, when the vehicle chooses a shorter path, it will receive positive rewards; when the vehicle is too close to obstacles or frequently changes lanes, it will be negatively punished. The reward function dynamically adjusts the importance of different indicators through weighting, allowing the agent to make reasonable decisions in different scenarios.
[0023] Step Four: Training the Agent and Optimizing the Path Strategy During training, the agent continuously tries different path choices in the simulated environment and adjusts its behavior strategy based on feedback from the reward function. To improve training stability, an experience replay mechanism and a double network structure (target network and evaluation network) are introduced. The former is used to store the agent's historical experience and randomly sample training, and the latter is used to separate policy evaluation and target update, reducing fluctuations and policy shocks during training.
[0024] Step Five: Executing Path Optimization Actions After training, the agent is deployed to the real vehicle system and outputs the optimal path actions based on the current environmental state. These actions include lane changes, acceleration, deceleration, detours, and parking, among others. The agent continuously adjusts the path based on real-time perception information to ensure the vehicle safely and efficiently reaches the destination in complex traffic environments.
[0025] Step Six: Path Optimization with High-precision Map and V2X Information To improve the accuracy and safety of path decision-making, the system accesses high-precision maps and V2X communication systems. High-precision maps provide static information such as road structure, speed limits, and traffic light locations, while V2X systems provide dynamic information such as upcoming accidents, construction road segments, and traffic signal states. The agent uses this information as part of the state input to adjust speed or path in advance, enabling no-stop intersection passing, improving traffic efficiency, and reducing energy consumption.
[0026] Step Seven: Support for Model Hot Updating and Continuous Optimization The system supports a model hot updating mechanism, which allows algorithm version upgrades without interrupting vehicle operation. By regularly downloading newly trained models from the cloud, vehicles can continuously adapt to new road environments and changes in traffic rules, ensuring that the path optimization strategy remains optimal at all times.
[0027] Implementation Process Two: Multi-vehicle Cooperative Path Optimization Process Step One: Build a Shared State Space Unlike the single-vehicle process, this process focuses on multi-vehicle cooperative path planning. Each vehicle collects its own state information through onboard sensors and shares information with surrounding vehicles through the V2X communication system. Shared information includes vehicle position, speed, target path, and path intent. The state information of all vehicles is integrated into a shared state space for use by the agents of each vehicle.
[0028] Step Two: Design a Joint Reward Mechanism To achieve multi-vehicle cooperative decision-making, a joint reward mechanism is designed. This mechanism considers not only the path optimization effect of individual vehicles but also the efficiency of the overall traffic flow. For example, when a vehicle chooses to change lanes to avoid another vehicle, both vehicles receive positive rewards; if multiple vehicles simultaneously choose the same path, leading to congestion, they will be penalized. In this way, vehicles are guided to coordinate their paths, avoiding conflicts and traffic congestion.
[0029] Step Three: Build a Cooperative Agent Network Each vehicle deploys an agent, but these agents share some strategy information through communication. The network structure of the agents is similar to that of Process One, using a combination of CNN+RNN+attention mechanism, but with added ability to perceive the path intent of other vehicles. The agent not only focuses on its own state but also predicts the behavior of other vehicles, making more reasonable path choices.
[0030] Step Four: Train Cooperative Path Planning Strategy During the training process, the agents conduct multi-vehicle coordination training in a simulated environment. Through experience replay and a dual network structure, the training stability is improved. In the early stage of training, two or three vehicles are mainly coordinated, and the number of vehicles is gradually increased to simulate real traffic scenarios. The training goal is to enable the agents to learn to make coordinated decisions with other vehicles in complex traffic environments, achieving path coordination and conflict avoidance.
[0031] Step five: Dynamic adjustment of path strategy In actual operation, the agent of each vehicle dynamically adjusts the path strategy according to the current environmental state and surrounding vehicle information. For example, when detecting a vehicle slowing down ahead, the agent can actively adjust the path, choosing to detour or slow down and wait, to avoid collisions and congestion. In addition, the agent can also adjust the vehicle speed in advance according to the traffic signal light state to realize no-stop passing through the intersection.
[0032] Step six: Access to cloud scheduling system for global optimization To further improve overall traffic efficiency, the system accesses the cloud scheduling platform. The cloud platform provides reference path suggestions for each vehicle based on global traffic flow data. The vehicle agent optimizes the individual path based on the reference path combined with local perception information, ensuring that the path selection meets both the overall traffic planning and individual driving needs.
[0033] Step seven: Support for continuous learning and strategy updating The system supports online learning mechanism, vehicles continuously collect new traffic data during operation and upload to the cloud for centralized training. The trained model is regularly updated to each vehicle, enabling the agent to continuously adapt to new traffic scenarios and improve the accuracy and efficiency of collaborative path planning.
[0034] Implementation process three: Multi-vehicle path intention conflict detection and reconciliation mechanism Step one: Path intention modeling The agent of each autonomous vehicle constructs a short-term path intention prediction model based on its state input sequence (position, speed, acceleration, historical path) and action output. This model uses a recurrent neural network (RNN) combined with a time attention mechanism to predict the path trend of each vehicle within the next few seconds, outputting a path intention vector .
[0035] Step two: Conflict detection function construction The system calculates the path intersection risk of each pair of vehicles based on the predicted path set of all vehicles : ; Where d ij is the minimum spatial distance between the predicted paths of two vehicles; Pconflict (i,j) is the overlap probability of the paths of two vehicles within the time window Δt; When C ij When the threshold τc is exceeded, the system determines that there is a potential path conflict.
[0036] Step three: path reconciliation mechanism After detecting a conflict, the system triggers the path reconciliation module. This module generates a set of reconciliation strategies based on the conflict intensity and chronological order, including: active deceleration avoidance strategy; slight lane adjustment strategy; predicted path time offset strategy; conflict area priority yielding strategy.
[0037] The agent adjusts the action output in real time according to the reconciliation strategy to ensure smooth path avoidance and travel.
[0038] Step four: conflict penalty feedback In the training process, to improve the safety and stability of multi-vehicle cooperative driving, the invention introduces a path conflict penalty term based on the original reward function, and the comprehensive reward function is defined as follows: ; Where R base represents the basic reward term, reflecting the travel efficiency and comfort; λ c is the conflict penalty coefficient, used to adjust the conflict constraint intensity; C ij is the path conflict coefficient between the i and j vehicles. When there is a risk of crossing or interference between multiple vehicle paths, the reward value automatically decreases, driving the agent to learn avoidance and coordination strategies, thereby achieving safety optimization of path planning in the reinforcement learning process.
[0039] Step five: system cooperative optimization Combined with V2X communication, vehicles share path intentions and reconciliation results in real time. The cloud or edge node periodically optimizes the global path distribution to avoid bottlenecks in the overall traffic flow.
[0040] Implementation process four: path optimization process based on curriculum learning and environment prediction Step one: build environment perception and prediction module This process emphasizes environmental prediction capabilities. Autonomous vehicles collect current traffic conditions through sensors and combine historical traffic data to build an environmental prediction module. This module uses time series prediction models and graph neural networks to model traffic flow, accident probability, road capacity, and other factors to predict traffic trends in the future.
[0041] Step two: design curriculum learning training strategy To enhance the agent's adaptability to complex scenarios, a curriculum learning strategy is employed. In the early stages of training, structured roads and low traffic density scenarios are prioritized, such as highways and urban main roads. As the training progresses, unstructured roads, pedestrian crossings, and sudden obstacles are gradually introduced. By gradually increasing the difficulty of training, the agent gradually acquires the ability to optimize paths in simple to complex scenarios.
[0042] Step three: Building a deep reinforcement learning model The agent adopts a CNN+RNN+attention mechanism structure, with CNN used to extract spatial features, RNN used to capture time series features, and attention mechanism used to dynamically adjust the importance of different features. The output of the environment prediction module is used as part of the state input, allowing the agent to avoid potential congestion or dangerous areas in path selection.
[0043] Step four: Designing a multi-objective reward function The reward function considers multiple indicators such as path length, travel time, obstacle avoidance distance, energy consumption, and passenger comfort. For example, the agent receives positive rewards when choosing a shorter path, and is penalized when it is too close to obstacles or frequently changes lanes. The reward function dynamically adjusts the importance of different indicators through weighting, allowing the agent to make reasonable decisions in different scenarios.
[0044] Step five: Training the agent and optimizing the path strategy During training, the agent continuously tries different path choices in the simulated environment and adjusts its behavior strategy based on the reward function feedback. To improve training stability, the experience replay mechanism and double network structure (target network and evaluation network) are introduced. The former is used to store the agent's historical experience and randomly sample training, and the latter is used to separate policy evaluation and target update, reducing fluctuations and policy shocks during training.
[0045] Step six: Executing path optimization actions After training, the agent is deployed to the real vehicle system and outputs the optimal path actions based on the current environmental state and prediction information. These actions include lane changes, acceleration, deceleration, detours, and parking waiting, etc. The agent continuously adjusts the path based on real-time perception information to ensure the vehicle safely and efficiently reaches the destination in complex traffic environments.
[0046] Step seven: Support model hot updating and continuous optimization The system supports model hot updating mechanism, which allows algorithm version upgrades without interrupting vehicle operation. By regularly downloading new trained models from the cloud, vehicles can continuously adapt to new road environments and changes in traffic rules, ensuring that the path optimization strategy remains optimal at all times.
[0047] Finally should be explained a few points are: first, in the description of the present application, it should be pointed out that, unless otherwise specified and limited, the term "installation", "connected", "connection" should be broad understanding, can be mechanical connection or electrical connection, can be two elements inside the communication, can be directly connected, "up", "down", "left", "right" and so on, only for indicating the relative position relationship, when the absolute position of the described object changes, the relative position relationship can change; Second: the present application discloses embodiment in the drawings, only involves the structure related to the present disclosure embodiment, other structures can refer to the usual design, in the case of no conflict, the same embodiment and different embodiments of the present application can be combined with each other; Finally: the above only for the preferred embodiment of the present application has, and does not for limiting the present application, any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application, should be included in the protection scope of the present application.
[0048] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made thereto without departing from the principles and spirit of the present application, the scope of the present application being defined by the appended claims and their equivalents.
Claims
1. An autonomous driving path optimization method based on deep reinforcement learning, characterized in that: Includes the following steps: By collecting real-time environmental state information of autonomous vehicles through onboard sensors and vehicle-to-everything (V2X) systems, a dynamic environmental state space is constructed. Based on this, a reinforcement learning agent based on deep neural networks is established to output path optimization actions according to the current environmental state, and the path planning strategy is continuously optimized through online training. In a multi-vehicle cooperative operation environment, the path intersection risk coefficient between any two vehicles is calculated based on the predicted path intentions of each vehicle. The agent dynamically adjusts the vehicle action strategy according to the conflict intensity and time sequence. The training process of the agent adopts a multi-objective reward function, which comprehensively considers path length, travel time, obstacle avoidance distance, energy consumption, and passenger comfort, and dynamically adjusts the reward weight. An experience replay mechanism and a dual network structure, including a target network and an evaluation network, are introduced during training to improve training stability and reduce policy oscillations. During the path optimization execution phase, the agent outputs the optimal action sequence based on the current environmental state, including lane changing, acceleration, deceleration, detour, and parking / waiting operations, to achieve real-time path optimization control of the autonomous vehicle.
2. The autonomous driving path optimization method based on deep reinforcement learning according to claim 1, characterized in that: The deep neural network adopts a structure that combines convolutional neural networks and recurrent neural networks. The convolutional neural network is used to extract the spatial distribution features of road images and obstacles, while the recurrent neural network is used to capture the time series features of traffic conditions and vehicle behavior. An attention mechanism is introduced to dynamically adjust the weights of environmental features, thereby enhancing the agent's ability to perceive complex environments and make path decisions.
3. The autonomous driving path optimization method based on deep reinforcement learning according to claim 1, characterized in that: A dynamic environment prediction module is set up during the route optimization process. Based on historical traffic data and current perception information, it predicts the future trend of traffic status changes. Combined with graph neural networks, it models traffic flow, accident probability and road capacity to improve the foresight of route planning and the ability to respond to emergencies.
4. The autonomous driving path optimization method based on deep reinforcement learning according to claim 1, characterized in that: The path optimization method supports multi-vehicle collaborative path planning. By sharing a state space and a joint reward mechanism, multiple autonomous vehicles can make collaborative decisions in complex traffic environments, achieving path coordination and conflict avoidance.
5. The autonomous driving path optimization method based on deep reinforcement learning according to claim 4, characterized in that: The path optimization method also includes a multi-vehicle path intent conflict detection and reconciliation mechanism. By constructing a path intent prediction model and a conflict detection function, the intersection probability and time overlap interval between the predicted paths of different vehicles are detected. When the conflict risk coefficient exceeds a preset threshold, the path reconciliation module is triggered. The agent dynamically adjusts the vehicle action strategy based on the conflict intensity and distance difference, including deceleration, detour, and lane change reservation, thereby realizing multi-vehicle path coordination and safe avoidance, and improving the overall stability and traffic flow efficiency of multi-vehicle collaborative path planning.
6. The autonomous driving path optimization method based on deep reinforcement learning according to claim 1, characterized in that: During training, a course learning strategy is introduced, gradually increasing the training difficulty according to the complexity of the environment from low to high, so that the agent can gradually master the path optimization ability from simple to complex scenarios, thereby improving the model's generalization performance and adaptability.
7. The autonomous driving path optimization method based on deep reinforcement learning according to claim 1, characterized in that: The route optimization strategy is linked with high-precision maps and V2X communication systems to acquire external data such as traffic light status, accident information ahead, and construction site locations in real time. This data is used as part of the status input to improve the accuracy, safety, and response speed of route decisions. The intelligent agent predicts the traffic light switching time based on traffic signals and adjusts the vehicle speed or route in advance to achieve non-stop passage through intersections, improving traffic efficiency and reducing energy consumption.
8. The autonomous driving path optimization method based on deep reinforcement learning according to claim 1, characterized in that: The path optimization method is deployed on the edge computing unit or cloud server of the autonomous driving system, supports real-time online updates and offline model iteration, and is applicable to autonomous vehicles of L2 to L4 levels. It can achieve efficient, safe and energy-saving path planning and control in different driving scenarios. The system supports a hot update mechanism for the model, which can complete the algorithm version upgrade without interrupting vehicle operation, ensuring that the path optimization strategy continuously adapts to changes in road environment and traffic rules.