Automatic driving decision planning method based on driver cognitive architecture
By introducing situational awareness and incremental learning methods based on driver cognitive architecture in autonomous vehicles, combined with graph neural networks and long-term memory networks, the shortcomings of the existing technology in complex scenario processing and interpretability are solved, and the improvement of the improvement of the problem of improving the problem of the new technology. The stability and self-learning ability of the decision-making system are achieved, and effective decision-making in different traffic scenarios are achieved.
Patent Information
- Application Number
- CN202411132317.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-08
AI Technical Summary
The decision-making methods of existing autonomous vehicles are poor in handling complex scenarios and lack interpretability, making them difficult to deploy effectively in the real world.
Using a decision planning method based on driver cognitive architecture, combined with situational awareness and incremental learning, we understand and predict the global relationship between traffic participants through graph neural networks and long-term memory networks, and construct and update an interpretable decision rule base. Improve the stability and self-learning ability of the decision-making system.
It improves the reliability and stability of the decision-making system of autonomous driving vehicles, realizes continuous learning and adaptation in different traffic scenarios, and ensures the interpretability and generalization capabilities of the system.
Smart Images

Figure QLYQS_2 
Figure QLYQS_4 
Figure BDA0004998287270000022
Abstract
Description
Technical Field
[0001] The present invention relates to the field of autonomous driving decision planning, and more specifically, to an autonomous driving decision planning method based on a driver cognitive architecture. Background Art
[0002] With the rapid development of artificial intelligence technology, autonomous driving technology has attracted extensive attention from scientific research institutes and related companies due to its advantages of improving driving safety, reducing traffic pressure and pollution, and has made significant progress. The vehicle's autonomous driving system includes three parts: environmental perception, decision planning and motion control. Among them, the decision system is an important part of autonomous driving technology. It is a bridge between environmental perception and motion control and is usually regarded as the "brain" of autonomous driving vehicles.
[0003] Among the existing decision-making methods for autonomous vehicles, the knowledge-driven method can handle simple scenarios well, but due to its insufficient ability to handle scene details, it performs poorly when handling complex scenarios; the data-driven method has self-learning capabilities and can learn to handle complex scene details through big data, thereby improving the accuracy of decision-making, but its generalization ability to adapt to scenarios is poor and it is not interpretable. With the deployment of autonomous vehicles in the real world, interpretability has become an important requirement for autonomous driving systems. Summary of the invention
[0004] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an autonomous driving decision-making planning method based on a driver cognitive architecture, and develops an autonomous driving vehicle decision model that incorporates situational awareness and incremental learning based on the driver cognitive architecture. While retaining the strong advantages of interpretability and generalization of the cognitive architecture, the model introduces the "perception, understanding and prediction" functions of situational awareness to comprehensively understand and analyze the global relationship between traffic participants, and improve the stability of the decision-making system caused by the insufficient perception ability of the original framework; it uses incremental learning to automatically extract interpretable rules from driving data, and automatically builds and updates the rule base, thereby improving its self-learning ability.
[0005] To achieve the above object, the present invention provides the following technical solution: an automatic driving decision planning method based on a driver cognitive architecture, comprising the following steps:
[0006] Step 1: Build a long-term memory library: Based on the driving experiment database, obtain the driving behavior decision rules under traffic difficulties;
[0007] Step 2: Use graph neural networks for scene understanding: Through graph convolution operations, the global relationship between traffic participants and other traffic participants in the scene can be obtained;
[0008] Step 3: Use LSTM for scene prediction: Use graph READOUT operation to generate spatial relationships for each time step t Then update the hidden state p based on the LSTM network t and the cell state c t ;
[0009] Step 4: Update the working memory bank: When the candidate decision obtained by matching is empty, the decision is obtained through incremental learning, and the long-term memory bank is updated and improved in real time online;
[0010] Step 5: Generation of decision instructions: When the candidate decisions obtained by search and matching are not empty, determine whether the decision is unique; if it is unique, directly output the decision, otherwise select the decision option with the largest Q value. Each rule has a corresponding Q value, which is updated by the Q-Learning algorithm;
[0011] Furthermore, in step 1, data corresponding to conditional attributes and decision attributes can be extracted from the driving experiment database, and the value of the conditional attribute can be used as the event cause in the fragment memory sentence, and the value of the discretized decision attribute can be used as the event result in the fragment memory sentence to obtain the fragment memory sentence.
[0012] Furthermore, the global relationship formula in step 2 is:
[0013]
[0014] Where: Represents the information transmission process at the lth layer of the network, N r (v) represents the set of adjacent targets of target v based on the relation r∈A, φ r It is a trainable specific transformation of relation r. Another trainable transformation φ_0 is used to eliminate the influence of the (l-1)th layer information on the lth layer. All h_v^((l)) are concatenated together to obtain H_v^L, which indicates that the traffic participant v has acquired the global relationship.
[0015] Furthermore, in step 3, the hidden state p is updated in the LSTM network t and the cell state c t The formula is:
[0016]
[0017] Furthermore, the update formula of the Q-Learning algorithm in step 5 is:
[0018] Q(s,a)←Q(s,a)+α[r+ymax a′ Q(s′,a′)-Q(s,a)#(7)
[0019] Where: s represents the current scene, s' represents the scene at the next moment, a represents the decision of the current scene, a' represents the decision applied to the next scene, r represents the reward obtained after applying the selected action, γ represents the attenuation factor applied to the reward, α is the learning rate, which represents the impact of the reward on the Q value, and the value range of γ and α is [0,1].
[0020] Technical effects and advantages of the present invention:
[0021] 1. The present invention introduces situational awareness into the driver's cognitive architecture, and uses graph neural networks and artificial potential fields to understand and predict surrounding traffic scenes, obtaining the "global" correlation relationship and future development trends that characterize all traffic participants, making the autonomous driving vehicle decision-making system more reliable and stable;
[0022] 2. The present invention uses incremental learning to extract interpretable rules from the driver behavior database and automatically builds the driver's cognitive rule base. Combined with incremental learning, it explores unfamiliar scenes while ensuring safety, learns new rules, and realizes online update of the rule base, so that the autonomous driving decision-making system can continuously learn and evolve to adapt to different traffic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 It is the overall flow chart of the present invention.
[0024] Figure 2 This is a generation diagram of new rules in the driver's cognitive structure of the present invention.
[0025] Figure 3 It is a diagram of the driver's cognitive decision-making process of the present invention.
[0026] Figure 4 This is a diagram for constructing the long-term memory bank of the present invention.
[0027] Figure 5 A diagram of the generation of decision instructions for the autonomous driving vehicle of the present invention and the updating of the working memory library. DETAILED DESCRIPTION
[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0029] according to Figure 1-5 The method for autonomous driving decision planning based on the driver cognitive architecture is characterized by comprising the following steps:
[0030] Step 1: Construct a long-term memory library: Based on the driving experiment database, obtain the driving behavior decision rules under traffic difficulties, extract the data corresponding to the condition attributes and decision attributes from the driving experiment database, use the value of the condition attribute as the event cause in the fragment memory sentence, and discretize the value of the decision attribute as the event result in the fragment memory sentence to obtain the fragment memory sentence;
[0031] Step 2: Use graph neural networks for scene understanding: Through graph convolution operations, we can obtain the global relationship between traffic participants and other traffic participants in the scene:
[0032]
[0033] Where: Represents the information transmission process at the lth layer of the network, N r (v) represents the set of adjacent targets of target v based on the relation r∈A, φ r It is a trainable specific transformation of relation r. Another trainable transformation φ_0 is used to eliminate the influence of the (l-1)th layer information on the lth layer. All h_v^((l)) are concatenated together to obtain H_v^L, which indicates that the traffic participant v has acquired the global relationship.
[0034] Step 3: Use LSTM for scene prediction: Use graph READOUT operation to generate spatial relationships for each time step t Then update the hidden state p based on the LSTM network t and the cell state c t :
[0035] Step 4: Update the working memory: When the candidate decision obtained by matching is empty, the decision is obtained through incremental learning, and the long-term memory is updated and improved in real time online. The specific process of incremental learning is as follows: Figure 2 As shown;
[0036] Step 5: Generation of decision instructions: When the candidate decisions obtained by search and matching are not empty, determine whether the decision is unique; if it is unique, directly output the decision, otherwise select the decision option with the largest Q value. Each rule has a corresponding Q value, which is updated by the Q-Learning algorithm:
[0037] Q(s,a)←Q(s,a)+α[r+ymax a′ Q(s′,a′)-Q(s,a)]#(7)
[0038] Where: s represents the current scene, s' represents the scene at the next moment, a represents the decision of the current scene, a' represents the decision applied to the next scene, r represents the reward obtained after applying the selected action, γ represents the attenuation factor applied to the reward, α is the learning rate, which represents the impact of the reward on the Q value, and the value range of γ and α is [0,1].
Claims
1. An autonomous driving decision-making planning method based on driver cognitive architecture, characterized in that: The following steps are involved: Step 1: Build a long-term memory library: Based on the driving experiment database, obtain the driving behavior decision rules under traffic difficulties; Step 2: Use graph neural networks for scene understanding: Through graph convolution operations, the global relationship between traffic participants and other traffic participants in the scene can be obtained; Step 3: Use LSTM for scene prediction: Use graph READOUT operation to generate spatial relationships for each time step t Then update the hidden state p based on the LSTM network t and the cell state c t ; Step 4: Update the working memory bank: When the candidate decision obtained by matching is empty, the decision is obtained through incremental learning, and the long-term memory bank is updated and improved in real time online; Step 5: Generation of decision instructions: When the candidate decisions obtained by search and matching are not empty, determine whether the decision is unique; if it is unique, directly output the decision, otherwise select the decision option with the largest Q value. Each rule has a corresponding Q value and is updated by the Q-Learning algorithm.
2. The method for autonomous driving decision-making and planning based on driver cognitive architecture according to claim 1, characterized in that: In the step 1, data corresponding to the conditional attributes and the decision attributes can be extracted from the driving experiment database, the value of the conditional attribute is used as the event cause in the fragment memory sentence, and the value of the discretized decision attribute is used as the event result in the fragment memory sentence, so as to obtain the fragment memory sentence.
3. The autonomous driving decision-making planning method based on the driver cognitive architecture according to claim 1, characterized in that: The global relationship formula in step 2 is: Where: Represents the information transmission process at the lth layer of the network, N r (v) represents the set of adjacent targets of target v based on the relation r∈A, φ r It is a trainable specific transformation of relation r. Another trainable transformation φ_0 is used to eliminate the influence of the (l-1)th layer information on the lth layer. All h_v^((l)) are concatenated together to obtain H_v^L, which indicates that the traffic participant v has acquired the global relationship.
4. The autonomous driving decision-making planning method based on the driver cognitive architecture according to claim 1, characterized in that: In step 3, the hidden state p is updated in the LSTM network. t and the cell state c t The formula is:
5. The method for autonomous driving decision-making and planning based on driver cognitive architecture according to claim 1, characterized in that: The update formula of the Q-Learning algorithm in step 5 is: Q(s,a)←Q(s,a)+α[r+γmax a' Q(s',a')-Q(s,a)]#(7) Where: s represents the current scene, s' represents the scene at the next moment, a represents the decision of the current scene, a' represents the decision applied to the next scene, r represents the reward obtained after applying the selected action, γ represents the attenuation factor applied to the reward, α is the learning rate, which represents the impact of the reward on the Q value, and the value range of γ and α is [0,1].
Citation Information
Cited By
Training method and device of automatic driving decision model, medium and product
CN120781921A
