Decision control method for unmanned vehicle complex scene based on deep reinforcement learning

Abstract

The invention relates to the technical field of intelligent driving, and discloses an unmanned vehicle complex scene decision control method and system based on deep reinforcement learning, and the method comprises the steps: extracting the multi-modal features of a traffic environment in real time through a graph attention network, and generating a scene descriptor; generating a decision strategy based on a dual-channel strategy generator, selecting a decision path according to a scene complexity evaluation result, calling a pre-decision cache library when the complexity is lower than a first threshold value, starting edge cooperative calculation when the complexity is higher than a second threshold value, and triggering cloud strategy generation when the complexity exceeds a third threshold value; and multi-target dynamic constraint verification is carried out on the generated candidate strategies, and execution strategies conforming to the safety boundary are screened and output to a vehicle control terminal. According to the method, the decision-making precision, the response real-time performance and the multi-objective optimization capability of the unmanned driving system in a complex traffic scene are remarkably improved, and meanwhile, efficient utilization of computing resources and functional safety compliance are realized.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a complex scenario decision-making and control method for unmanned vehicles based on deep reinforcement learning. Background Art

[0002] With the rapid development of artificial intelligence (AI), deep reinforcement learning (DRL) autonomous driving technology has made significant progress in recent years. Driven in particular by deep learning and reinforcement learning algorithms, autonomous vehicles have significantly improved their decision-making and control capabilities in complex traffic environments. Traditional autonomous driving systems typically rely on pre-set rules and path planning algorithms. While these methods perform well in simple scenarios, they often suffer from slow decision-making and insufficient response capabilities in dynamic and complex traffic environments. Advances in sensor technology allow vehicles to obtain real-time information about their surroundings, such as traffic signs, the status of pedestrians, and other vehicles. This allows deep learning models to extract high-dimensional feature data, enabling more intelligent decision-making. In particular, the introduction of graph neural networks (GNNs) and attention mechanisms has enabled autonomous driving systems to effectively integrate multimodal information, further enhancing their adaptability in complex scenarios.

[0003] However, current research and applications still have obvious shortcomings. Many existing autonomous driving decision-making frameworks focus too much on static features when dealing with complex scenarios, ignoring the dynamic changes in the traffic environment. The black box nature of traditional end-to-end models leads to poor interpretability of the decision-making process. The existing DRL training framework has limited coverage of real-world scenarios, especially in long-tail scenarios (such as the sudden appearance of wrong-way vehicles on construction sections), where the risk of strategy failure increases significantly, and cannot meet the real-time requirements of complex urban scenarios. These problems seriously restrict the reliability and universality of unmanned driving systems on open roads. Therefore, how to effectively extract and integrate multimodal features, dynamically evaluate complex scenarios and generate safe and reliable decision-making strategies has become an important challenge that needs to be urgently addressed in current unmanned driving technology. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a complex scenario decision-making and control method for unmanned vehicles based on deep reinforcement learning, which can solve the problems of interpretability and functional safety compliance of traditional DRL decision models in complex scenarios and the insufficient strategy generalization ability caused by the sparsity of real driving data.

[0006] To solve the above technical problems, the present invention provides the following technical solutions. A complex scenario decision and control method for driverless vehicles based on deep reinforcement learning includes: real-time extraction of multi-modal features of the traffic environment through a graph attention network to generate a structured scenario descriptor containing spatio-temporal semantic relationships; generation of decision-making strategies based on a dual-channel policy generator, where the main channel loads a pre-trained base model, and the auxiliary channel dynamically adjusts the parameters of the base model through meta-reinforcement learning; selection of a decision-making path according to the scenario complexity evaluation result, calling a pre-decision cache library when the complexity is lower than the first threshold, starting edge collaborative computing when it is higher than the second threshold, and triggering cloud policy generation when it exceeds the third threshold; multi-objective dynamic constraint verification of the generated candidate policies, and screening the execution policies that meet the safety boundaries and outputting them to the vehicle control terminal.

[0007] As a preferred solution of the complex scenario decision and control method for driverless vehicles based on deep reinforcement learning according to the present invention, wherein: the structured scenario descriptor includes constructing traffic elements into graph data containing node attributes and topological relationships, calculating the attention weights of spatially adjacent nodes, temporally consecutive frames, and semantically associated elements respectively by using a third-order attention mechanism, and generating a descriptor vector by fusing multi-dimensional features through graph convolution operations; The third-order attention mechanism includes a spatial attention layer for calculating the Euclidean distance weights of adjacent traffic elements; a temporal attention layer for analyzing the dynamic change trends of historical trajectory points; and a semantic attention layer for identifying the priority relationships under traffic rule constraints.

[0008] As a preferred solution of the complex scenario decision and control method for driverless vehicles based on deep reinforcement learning according to the present invention, wherein: the dual-channel policy generator includes loading a base model pre-trained with real driving data on the main channel and keeping the network parameters frozen; the auxiliary channel receiving the current scenario descriptor and generating an amount for adjusting the parameters of the base model; and generating a final decision-making strategy by superimposing the adjustment amount on the base model parameters through a parameter interpolation algorithm.

[0009] As a preferred solution of the complex scenario decision and control method for driverless vehicles based on deep reinforcement learning according to the present invention, wherein: the scenario complexity evaluation includes extracting the number of dynamic obstacles, relative speed variance, and road structure complexity indicators in the scenario descriptor, and classifying the scenario into four levels L1-L4 by using a lightweight classifier; When the evaluation is L1, call the policy corresponding to the matching scenario feature identifier in the local cache policy library; when it is L2, exchange the scenario feature encoding with the edge server and receive the optimized policy; when it is L3, request the cloud to generate a new policy; when it is L4, enable the safety redundancy policy and request manual takeover.

[0010] As a preferred embodiment of the complex scenario decision-making and control method for driverless vehicles based on deep reinforcement learning according to the present invention, wherein: the multi-objective dynamic constraint verification includes establishing a three-dimensional evaluation space including safety distance evaluation, energy consumption efficiency evaluation, and comfort evaluation, generating a Pareto front solution set of candidate strategies through online optimization, and filtering strategies that exceed the preset safety boundary using a confidence threshold.

[0011] As a preferred embodiment of the complex scenario decision-making and control method for driverless vehicles based on deep reinforcement learning according to the present invention, wherein: the safety distance evaluation includes generating a safety score based on the real-time distance and relative speed between the vehicle and the obstacle using a preset time-to-collision calculation formula; The energy consumption efficiency evaluation includes accessing the estimated time of arrival data of the vehicle navigation and the instantaneous energy consumption data of the battery management, and calculating the time-energy comprehensive efficiency value; The comfort evaluation includes collecting the longitudinal acceleration change rate and lateral swing amplitude through an in-vehicle inertial measurement unit, and setting an acceleration change threshold for quantitative evaluation; Perform a weighted sum on the three-dimensional evaluation results of the candidate strategies, and select the strategy with the highest total score and a qualified safety dimension, where the weight distribution is dynamically adjusted according to the current driving mode.

[0012] As a preferred embodiment of the complex scenario decision-making and control system for driverless vehicles based on deep reinforcement learning according to the present invention, wherein: it includes an environment perception module, a decision generation module, a complexity evaluation module, a strategy verification module, and an execution control module;

[0013] The environment perception module is responsible for acquiring and processing data from sensors to form a comprehensive understanding of the surrounding environment; The decision generation module is responsible for generating driving strategies for the driverless vehicle in different scenarios; The complexity evaluation module evaluates the complexity of the scenario and calls different decision-making strategies according to the complexity level; when the scenario complexity reaches the set threshold, it decides whether to call the local strategy library, edge server, or cloud to generate new strategies; The strategy verification module performs multi-objective dynamic constraint verification on the generated candidate strategies. Generate feasible strategies that meet the safety boundary through online optimization, and filter out strategies that do not meet the requirements using a confidence threshold; The execution control module controls the driving of the vehicle according to the selected safety strategy. The execution control module controls the driving of the vehicle according to the selected safety strategy.

[0014] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of a complex scenario decision control method for an autonomous vehicle based on deep reinforcement learning are implemented.

[0015] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a complex scenario decision control method for an autonomous vehicle based on deep reinforcement learning are implemented.

[0016] Advantages of the present invention: Through multi-modal feature fusion and a third-order graph attention mechanism, spatio-temporal semantic joint encoding of traffic elements is achieved, effectively improving the parsing accuracy and interpretability of complex scenarios; A dual-channel architecture is adopted to achieve rapid adaptation of new scenario strategies while retaining large-scale pre-trained knowledge; A hierarchical resource scheduling mechanism based on scenario complexity breaks through the single-vehicle computing power bottleneck through the organic linkage of local cache reuse, edge collaborative computing, and cloud elastic expansion, achieving dynamic balance of computing load while ensuring real-time performance and extending the service life of in-vehicle hardware; A three-dimensional evaluation space integrating safety, efficiency, and comfort is constructed, combined with Pareto front optimization and dynamic constraint adjustment mechanisms, breaking through the sub-optimal dilemma of traditional single-objective optimization, and achieving intelligent trade-off between multiple objectives and adaptive expansion and contraction of safety boundaries in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0018] Figure 1 It is a schematic flowchart of a complex scenario decision control method for an autonomous vehicle based on deep reinforcement learning provided by an embodiment of the present invention.

[0019] Figure 2 It is a schematic diagram of the working modules of a complex scenario decision control system for an autonomous vehicle based on deep reinforcement learning provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following provides a detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0022] Secondly, the so-called "one embodiment" or "embodiment" herein refers to specific features, structures, or characteristics that may be included in at least one implementation manner of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that mutually excludes other embodiments.

[0023] The present invention is described in detail in conjunction with schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views showing the device structure are enlarged locally not in accordance with the general scale, and the schematic diagrams are only examples and should not limit the scope of protection of the present invention herein. In addition, in actual production, three-dimensional spatial dimensions including length, width, and depth should be included.

[0024] At the same time, in the description of the present invention, it should be noted that the orientation or positional relationships indicated by terms such as "upper, lower, inner, and outer" are based on the orientation or positional relationships shown in the drawings. This is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of the present invention. In addition, the terms "first, second, or third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0025] Unless otherwise clearly defined and limited in the present invention, the terms "installed, connected, and coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can also be a mechanical connection, an electrical connection, or a direct connection, or can be indirectly connected through an intermediate medium, or can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0026] Example 1, referring to Figure 1 , which is the first embodiment of the present invention. This embodiment provides a decision-making control method for complex scenarios of driverless vehicles based on deep reinforcement learning, including: S1: Real-time extract multimodal features of the traffic environment through a graph attention network to generate a structured scene descriptor containing spatio-temporal semantic relationships.

[0027] Construct a dynamic traffic element map through the fused perception data of on-vehicle lidar, cameras, and millimeter-wave radars; among them, each traffic element node contains coordinate, speed, and type coding attributes, and the topological connection edges contain relative distance and moving direction angle attributes.

[0028] Furthermore, to construct a dynamic traffic element map, collect the position coordinates, moving speeds, and type labels of surrounding obstacles through on-vehicle sensors, model each traffic element as a graph node, and the node attribute vector contains coordinates , speed , type coding (vehicle = 01, pedestrian = 02, traffic light = 03); establish topological connection edges according to the relative position relationship between elements, and the edge attributes include Euclidean distance, relative speed, and interaction relationship coding (same lane = 1, cross path = 2); Furthermore, the third-order attention mechanism includes that the spatial attention layer calculates the Euclidean distance weights of adjacent traffic elements; the temporal attention layer analyzes the dynamic change trend of historical trajectory points; and the semantic attention layer identifies the priority relationship under traffic rule constraints.

[0029] It should be noted that the spatial attention layer establishes a polar coordinate system centered on the vehicle itself, calculates the relative azimuth angles of each node within a radius of 50 meters, assigns basic weights according to the node type (exemplarily, vehicle = 0.6, pedestrian = 0.8, static obstacle = 0.4), and dynamically corrects the weights in combination with the angle between the node moving direction and the vehicle path; for each traffic element node, calculate its moving direction vector through the coordinate changes of consecutive frames, predict the future path direction of the vehicle based on the current vehicle state and high-precision map data, calculate the angle between the node moving direction vector and the vehicle path vector, and divide the angle range into three intervals: same direction (0° - 30°), cross (30° - 150°), opposite (150° - 180°); when it is a same-direction moving node, reduce the attention weight; when it is a cross-moving node, increase its attention weight; when it is an opposite-moving node, assign the highest priority weight.

[0030] The temporal attention layer extracts the historical trajectory points of each node within the past 3 seconds (exemplarily, sampling frequency 10Hz), calculates the acceleration change amount of each trajectory point through the first-order difference method, and statistically calculates the trajectory smoothness index using a sliding window method; the semantic attention layer accesses high-precision map data to obtain traffic rule constraints for the current road section, adjusts the node weights according to the right-of-way priority (such as increasing the pedestrian weight by 30% in the crosswalk area), and activates turning conflict detection in the intersection scenario.

[0031] The three types of attention weights are gated and added together and input into a three-layer graph convolutional network (GCN). Each layer uses the ReLU activation function and implements edge dropout regularization (for example, edge dropout=0.2). The final output is a feature vector of dimension 256 as the scene descriptor.

[0032] S2: Decision strategy generation is performed based on a dual-channel strategy generator, where the primary channel loads the pre-trained base model, and the secondary channel dynamically adjusts the base model parameters through meta-reinforcement learning.

[0033] Furthermore, the main channel architecture deploys a convolutional neural network model pre-trained on massive amounts of real-world driving data. This model consists of multiple residual modules and uses a global feature pooling layer to keep network parameters fixed during the policy generation phase. It uses a ResNet-18 variant with four residual blocks (two 3×3 convolutional layers per block), replaces fully connected layers with global average pooling layers, and outputs a 256-dimensional policy distribution vector. The auxiliary channel constructs a meta-learning parameter adjustment network, where the network architecture of the parameter adjustment network includes: the input layer is a 256-dimensional scene descriptor; the hidden layer is a 2-layer LSTM (128 units); the output layer dynamically generates channel-level convolution kernel scaling factors, spatial-level attention masks, and feature selection gates for the fully connected layer; After receiving the current scene descriptor, an adjustment parameter matrix is generated for each network layer of the main channel model. The dimension of the adjustment parameter matrix is consistent with the weight dimension of the corresponding layer of the main channel model. The scene descriptor vector is input to a two-layer fully connected network (hidden layer dimension 128→64). The output adjustment matrix is restructured according to the dimension of the base model convolution layer, and an L2 norm constraint is applied to the adjustment amount to prevent parameter mutation. The parameter fusion process implements channel-level scaling adjustment on the convolution layer of the main channel model. Specifically, it decomposes the adjustment parameter matrix into a channel attention vector and a spatial filter template. The channel attention vector is used to adjust the contribution weight of each feature channel, and the spatial filter template is used to correct the spatial response characteristics of the convolution kernel. The adjustment matrix is decomposed into a channel scaling factor and a spatial filter template. Channel-level scaling and spatial domain filtering operations are performed on the convolution layer of the base model, and 1 / 4 of the base weight of the original convolution kernel is retained as a bottom parameter. A feature gating mechanism is used to adjust the parameters of the fully connected layer. The original features and the modified features are dynamically mixed through the selection coefficient vector generated by the meta-network to retain the core feature expression ability of the basic model.

[0034] It should be noted that the feature selection mechanism of the fully connected layer is to generate a binary gating vector to control the activation state of the feature path. For feature dimensions with a gating value lower than 0.3, the original output of the base model is maintained, and for feature dimensions with a gating value higher than 0.7, the meta-network correction output is enabled.

[0035] S3: Select a decision path based on the evaluation result of the scenario complexity. When the complexity is lower than the first threshold, call the pre - decision cache library; when it is higher than the second threshold, start edge collaborative computing; when it exceeds the third threshold, trigger cloud - side policy generation; when an emergency event is detected, enable the preset security redundancy policy.

[0036] Furthermore, extract the number of dynamic obstacles, relative speed variance, and road structure complexity metrics in the scenario descriptor, and use a lightweight classifier to divide the scenario into four levels: L1 - L4; Among them, the number of dynamic obstacles , count the total number of nodes with type encoding as dynamic targets in the descriptor; relative speed variance , calculate the relative speed variance value between all dynamic nodes and the vehicle; road structure complexity , calculate the curvature integral value of the current road section based on the high - precision map data; use a lightweight XGBoost classifier for level division, and the input features of the classifier are , and output the four - level determination result: Level L1: and and ; Level L2: or or ; Level L3: or or ; Level L4: Detect the construction area sign or the approaching signal of an emergency vehicle; When the evaluation is L1, perform scenario feature identifier matching in the local cache policy library, and use the locality - sensitive hashing algorithm for fast retrieval, with the matching threshold set as Hamming distance ≤ 3; when it is L2, send a 128 - bit scenario feature encoding (MD5 hash value) to the edge server, receive the optimized policy from the server and update the local cache, when it is L3, trigger the cloud - side policy generation service, use containerized instances to generate candidate policies in parallel, and after the generation result is verified, send it back to the in - vehicle terminal, when it is L4, enable the preset security redundancy policy and simultaneously send a driver takeover request, and the takeover request is transmitted through the dual channels of the HMI interface and voice alarm.

[0037] It should be noted that when the cone feature of the construction section is detected for 5 consecutive frames, the scenario level is forcibly raised to L3; under rain and snow weather conditions, an additional 20% safety margin check is added to the L1 - L3 level policies; for a scenario evaluated as L1 for 10 consecutive frames, the feature extraction frequency is reduced from 10Hz to 5Hz; when the response delay of the edge server exceeds 200ms, automatically switch to the in - vehicle backup policy generation mode.

[0038] S4: Perform multi-objective dynamic constraint verification on the generated candidate strategies, and filter out the execution strategies that meet the safety boundaries and output them to the vehicle control terminal.

[0039] Establish a three-dimensional evaluation space including safety distance, energy consumption efficiency, and comfort. Align the data through time synchronization, and perform standardization processing on the original indicators of each dimension to eliminate the dimension differences. Use the spatial discretization method to divide the continuous evaluation domain into regular grid cells, and each cell records the strategy features and evaluation metadata; establish a dynamically updated spatial index structure to support efficient neighborhood queries and range retrievals. Generate the Pareto front solution set of candidate strategies through online optimization, and use the confidence threshold to filter out the strategies that exceed the preset safety boundaries.

[0040] Furthermore, the safety distance evaluation uses the hierarchical time-to-collision calculation. When the obstacle is in the ±30-degree area directly in front of the vehicle, the time-to-collision with deceleration compensation is used, and a safety factor of 1.2 is added to the laterally moving obstacle. Under rainy or foggy weather conditions, the visibility attenuation factor is used to correct the calculated value; The energy consumption efficiency evaluation integrates vehicle data, obtains the remaining mileage and road speed limit data from the path planning, and obtains the real-time energy consumption rate (kWh / km) from the power system, and calculates the energy consumption ratio under standard conditions as the efficiency scoring benchmark; The comfort evaluation implements multi-level threshold control. When the longitudinal acceleration change rate exceeds 0.3g, a first-level warning is triggered. When the lateral swing amplitude continues to exceed 0.2g for 3 seconds, the trajectory smoothing correction is started, and the deceleration gradient limit algorithm is enabled for continuous braking scenarios; Perform weighted summation on the three-dimensional evaluation results of the candidate strategies, and select the strategy with the highest total score and the safety dimension meeting the standards, where the weight distribution is dynamically adjusted according to the current driving mode.

[0041] Furthermore, for the dynamic adjustment of the safety boundary, the visibility index (0-1 standardized value) obtained by the on-vehicle camera is used to calculate the basic safety distance value × (1 + visibility compensation coefficient), and the compensation coefficient increases exponentially as the visibility decreases; Perform lateral control constraints through road surface condition perception. Connect to the tire pressure monitoring unit to obtain the tire ground pressure distribution, calculate the maximum allowable lateral acceleration according to the pressure distribution uniformity, and automatically reduce the upper limit of the steering angular velocity in the waterlogged road surface scenario; Perform comfort adjustment according to the occupant status feedback. Detect the occupant posture stability through the seat pressure sensor. When frequent posture adjustments are detected, relax the acceleration change rate constraint, and enable enhanced safety filtering conditions in the child riding mode.

[0042] Embodiment 2, the second embodiment of the present invention, which is different from the previous embodiment in that: When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0043] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a defined sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or used in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0044] More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection parts with one or more wirings (electronic devices), portable computer disk cartridges (magnetic devices), random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memories), fiber optic devices, and portable compact disc read-only memories (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, then editing, interpreting, or otherwise processing it as necessary, and then storing it in a computer memory.

[0045] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0046] Example 3. Referring to Figure 2 , which is an embodiment of the present invention, provides a complex scenario decision control system for driverless vehicles based on deep reinforcement learning, including an environmental perception module, a decision generation module, a complexity evaluation module, a policy verification module, and an execution control module; The environmental perception module is responsible for acquiring and processing data from sensors to form a comprehensive understanding of the surrounding environment; The decision generation module is responsible for generating driving strategies for driverless vehicles in different scenarios; The complexity evaluation module evaluates the complexity of the scenario and calls different decision-making strategies according to the complexity division; when the scenario complexity reaches a set threshold, it decides whether to call the local policy library, the edge server, or the cloud to generate a new strategy; The policy verification module performs multi-objective dynamic constraint verification on the generated candidate policies. Through online optimization, a feasible policy that meets the safety boundary is generated, and policies that do not meet the requirements are filtered using a confidence threshold; The execution control module controls the driving of the vehicle according to the selected safety policy.

[0047] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A decision-making control method for driverless vehicles in complex scenarios based on deep reinforcement learning, characterized in that: including real-time extraction of multi-modal features of the traffic environment through a graph attention network to generate a structured scene descriptor containing spatio-temporal semantic relationships generating a decision-making strategy based on a dual-channel policy generator, where the main channel loads a pre-trained base model, and the auxiliary channel dynamically adjusts the parameters of the base model through meta-reinforcement learning selecting a decision-making path according to the scene complexity evaluation result, calling the pre-decision cache library when the complexity is lower than the first threshold, starting edge collaborative computing when it is higher than the second threshold, and triggering cloud policy generation when it exceeds the third threshold performing multi-objective dynamic constraint verification on the generated candidate policies, and screening the execution policies that meet the safety boundaries and outputting them to the vehicle control terminal 2. The decision control method for complex scenarios of driverless vehicles based on deep reinforcement learning according to claim 1, wherein: The structured scene descriptor includes constructing traffic elements into graph data containing node attributes and topological relationships, calculating the attention weights of spatially adjacent nodes, temporally consecutive frames, and semantically related elements using a third-order attention mechanism, and generating a descriptor vector by fusing multi-dimensional features through graph convolution operations The third-order attention mechanism includes a spatial attention layer that calculates the Euclidean distance weights of adjacent traffic elements a temporal attention layer that analyzes the dynamic change trend of historical trajectory points a semantic attention layer that identifies the priority relationships under traffic rule constraints 3. The decision-making and control method for complex scenarios of a driverless vehicle based on deep reinforcement learning according to claim 2, characterized in that: The dual-channel policy generator includes a main channel that loads a base model pre-trained with real driving data and keeps the network parameters frozen; the auxiliary channel receives the current scene descriptor and generates an adjustment amount for the parameters of the base model; the adjustment amount is superimposed on the base model parameters through a parameter interpolation algorithm to generate the final decision-making strategy 4. The decision-making control method for complex scenarios of a driverless vehicle based on deep reinforcement learning according to claim 3, characterized in that: The scene complexity evaluation includes extracting the number of dynamic obstacles, relative speed variance, and road structure complexity indicators in the scene descriptor, and dividing the scene into four levels of L1-L4 using a lightweight classifier When evaluated as L1, call the policy corresponding to the scene feature identifier in the local cache policy library; when it is L2, exchange the scene feature encoding with the edge server and receive the optimized policy; when it is L3, request the cloud to generate a new policy; when it is L4, enable the safety redundancy policy and request manual takeover 5. The decision control method for complex scenarios of a driverless vehicle based on deep reinforcement learning according to claim 4, characterized in that: The multi-objective dynamic constraint verification includes establishing a three-dimensional evaluation space including safety distance evaluation, energy consumption efficiency evaluation, and comfort evaluation, generating a Pareto front solution set of candidate policies through online optimization, and filtering out policies that exceed the preset safety boundaries using a confidence threshold 6. The decision-making and control method for complex scenarios of driverless vehicles based on deep reinforcement learning according to claim 5, characterized in that: The safety distance evaluation includes generating a safety score based on the real-time distance and relative speed between the vehicle and the obstacle using a preset time-to-collision calculation formula The energy consumption efficiency evaluation includes accessing the estimated arrival time data of the vehicle navigation and the instantaneous energy consumption data of the battery management, and calculating the time-energy comprehensive efficiency value The comfort evaluation includes collecting the longitudinal acceleration change rate and lateral swing amplitude through an in-vehicle inertial measurement unit, and setting an acceleration change threshold for quantitative evaluation Perform weighted summation on the three-dimensional evaluation results of the candidate policies, and select the policy with the highest total score and qualified safety dimension, where the weight distribution is dynamically adjusted according to the current driving mode 7. The decision-making control method for complex scenarios of driverless vehicles based on deep reinforcement learning according to claim 6, characterized in that: The safety boundary includes: dynamically shrinking the longitudinal safety distance threshold according to the environmental visibility, adjusting the maximum allowable lateral acceleration based on the road surface adhesion coefficient, and relaxing the comfort constraint range in combination with the passenger status monitoring data.

8. A system adopting the decision-making control method for complex scenarios of a driverless vehicle based on deep reinforcement learning as described in any one of claims 1 to 7, characterized in that: It includes an environmental perception module, a decision generation module, a complexity evaluation module, a policy verification module, and an execution control module; The environmental perception module is responsible for acquiring and processing data from sensors to form a comprehensive understanding of the surrounding environment; The decision generation module is responsible for generating driving strategies for the driverless vehicle in different scenarios; The complexity evaluation module evaluates the complexity of the scenario and calls different decision-making strategies according to the complexity classification; When the scenario complexity reaches the set threshold, it decides whether to call the local policy library, the edge server, or the cloud to generate a new strategy; The policy verification module conducts multi-objective dynamic constraint verification on the generated candidate strategies, generates a feasible strategy that meets the safety boundary through online optimization, and filters out strategies that do not meet the requirements using a confidence threshold; The execution control module controls the driving of the vehicle according to the selected safety strategy.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method described in any one of claims 1 to 7.

Images